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ABSTRACT 


For decades, the United States’ major defense acquisition system has been under 
scrutiny and undergone much reform. Groups have researched the issues, publishing 
hundreds of reports identifying various problems and solutions. Yet, many major weapon 
systems continue to be well over budget and schedule. Major weapon systems are 
increasing in size, scope, and complexity. Technology is rapidly changing. Customer 
expectations are rising. Societal concerns, such as workforce and economic 
development, are playing a bigger role. Politics are rampant in this system. This system 
qualifies as a CLIOS system—Complex, Large-scale, Interconnected, Open, and 
Sociotechnical in nature. 

This thesis explored and analyzed the decades of research concerning U.S. major 
weapon systems acquisitions and applied the CLIOS Process. Of the three stages within 
the CLIOS Process, this research applied the Representation Stage to the U.S. major 
defense acquisition system. The observations afforded from the analysis were: (1) long¬ 
term decisions are made with short-term information and (2) multiple stakeholders and 
decision makers facilitate little accountability. Three strategic alternatives were 
identified: (1) create an Integrated Process Team to make joint long-term decisions, (2) 
mandate a Federal Systems Engineering organization, and (3) create a hybrid between 
the first two for instilling accountability at all levels. 


V 



THIS PAGE INTENTIONALLY LEET BLANK 


VI 



TABLE OF CONTENTS 


I. INTRODUCTION.I 

A. BACKGROUND.I 

B. PURPOSE AND BENEFITS OF STUDY.2 

C. RESEARCH QUESTIONS.3 

D. METHODOLOGY.3 

II. REVIEW OF LITERATURE.5 

A. INTRODUCTION.5 

B. ISSSUES SURROUNDING MAJOR WEAPON SYSTEMS 

ACQUISITION.5 

C. REASONS AND RECOMMENDATIONS.10 

D. INITIATIVES TO IMPROVE ACQUISITION OF MAJOR 

WEAPON SYSTEMS.16 

E. TOP MILITARY SPENDING COUNTRIES.20 

1. The United States’ Defense Acquisition System.21 

2. France’s Defense Acquisition System.24 

3. Germany’s Defense Acquisition System.27 

4. The United Kingdom’s Defense Acquisition System.29 

5. Australia’s Defense Acquisition System.31 

6. Japan’s Defense Acquisition System.33 

7. South Korea’s Defense Acquisition System.36 

III. METHODOLOGY.39 

A. CLIOS PROCESS.39 

B. CLIOS STRUCTURE.40 

1. Representation Stage.43 

2. Design, Evaluation, and Selection Stage.46 

3. Implementation Stage.48 

C. CONCLUSION.49 

IV. CLIOS STAGE I REPRESENTATION—MAJOR DEFENSE 

ACQUISTION SYSTEM.51 

A. INTRODUCTION.51 

B. STEP I—DESCRIBE THE SYSTEM: CHECKLISTS AND 

PRELIMINARY GOAL IDENTIFICATION.51 

I. Characteristics Checklist.52 

a. What is the Temporal and Geographic Scale of the 

System? . 52 

b. What are the Core Technologies and Systems? . 53 

c. What are the Key Economic and Market Issues? . 54 

d. Are There any Important Social or Political Issues or 

Controversies that Relate to this System? . 54 

vii 




































e. What is the Historical Development and Context of the 

Major Defense Acquisition System? . 55 

f What are the Policy Questions that Need to be Addressed? ..55 

2. Opportunities, Issues, and Challenges Checklist.56 

3. Preliminary Major Defense Acquisition System Goals 

Checklist.57 

C. STEP 2—IDENTIFY SUBSYSTEMS IN PHYSICAL DOMAIN & 

GROUPS ON INSTITUTIONAL SPHERE.57 

D. STEP 3—POPULATE THE PHYSICAL DOMAIN & THE 

INSTITUTIONAL SPHERE.60 

E. STEP 4—DESCRIBE COMPONENTS AND LINKS.63 

1. Physical Domain—Planning, Programming, Budgeting, and 

Execution (PPBE).63 

2. Physical Domain—Joint Capabilities Integration and 

Development System (JCIDS).63 

3. Physical Domain—Defense Acquisition System.64 

4. Institutional Sphere—Congress.64 

5. Institutional Sphere—Department of Defense.65 

6. Institutional Sphere—The Military.65 

7. Institutional Sphere—The Executive Branch.65 

8. Institutional Sphere—Industry.66 

9. Institutional Sphere—The U.S. Public.66 

10. Institutional Sphere—Security Threats.67 

F. CLIOS ANALYSIS.67 

1. Observation I: Long-term Decisions—Short-term Information ..68 

2. Solution I: Integrated Process Team.69 

3. Solution II: Mandated Systems Engineering Organization.71 

4. Observation II: Working in Silos—Little Accountability.72 

5. Solution III: Defense Acquisition Accountability.73 

G. CONCLUSION.75 

V. CONCLUSION.77 

A. CONCLUSION.77 

B. SUMMARY OF KEY POINTS.77 

C. LESSONS LEARNED / HEURISTICS.78 

D. RECOMMENDATIONS AND FURTHER RESEARCH.79 

LIST OF REFERENCES.81 

INITIAL DISTRIBUTION LIST.85 


viii 
































LIST OF FIGURES 


Figure 1. Typical Flow of Budget [From: Kausal, 1999].22 

Figure 2. The Acquisition System [From: Defense Acquisition Performance 

Assessment Report, 2006, p. 4].23 

Figure 3. Defense Management Acquisition System [From: DoDI 5000.02, 2008].24 

Figure 4. Organization of National Defense [From: Kausal, 1999].25 

Figure 5. New Acquisition Process.26 

Figure 6. Weapon Systems Development Process [From: Kausal, 1999, p. 2-21].28 

Figure 7. The U.K.’s Defence Procurement Phases [From: Kausal, 1999, p. 3-24].30 

Figure 8. Defense Acquisition Organizational Structure [From: Kausal, 2000, p. 1- 

41].32 

Figure 9. Australia’s Acquisition Life-cycle [From: Kausal, 2000, p. 1-49].33 

Figure 10. Japan’s Defense Agency [From: Kausal, 2000, p. 2-15].34 

Figure 11. Japan’s Acquisition System [From: Kausal, 2000, p. 2-26].35 

Figure 12. The Republic of Korea’s Ministry of National Defense [From: Kausal, 

2000, p. 3-4].36 

Figure 13. Korea’s Acquisition Development Structure [From: Kausal, 2000, p. 3- 

17].37 

Figure 14. Korean Ministry of National Defense Acquisition Process [From: Kausal, 

2000, p. 3-27].38 

Figure 15. CLIOS System—Physical Domain with Subsystems Embedded in an 

Institutional Sphere.40 

Figure 16. CLIOS Process (From: Sussman, 2007, p. 15).42 

Figure 17. CLIOS System Checklists (Lrom: Sussman, 2007, p. 20).43 

Ligure 18. Suggested CLIOS System Diagram Component Shapes (Lrom: Sussman, 

2007, p. 24).45 

Ligure 19. Suggested Link Shapes (Lrom: Sussman, 2007, p. 27).46 

Ligure 20. Iteration in the CLIOS Process (Prom: Sussman, 2007, p. 16).50 

Ligure 21. Temporal Scale.52 

Ligure 22. Geographic Scale.53 

Ligure 23. Core Technologies and Systems.53 

Ligure 24. Integrated Defense Acquisition (Prom: DAU website, 

https://acc.dau.mil/ifc/).59 

Ligure 25. Major Defense Acquisition System—Layering of Subsystems and 

Institutional Sphere.61 

Ligure 26. Major Defense Acquisition System—Links Between Institutional Sphere 

and Physical Domain.62 

Ligure 27. CLIOS Representation: Long-term Decisions - Short-term Information.69 

Ligure 28. Integrated Process Team.70 

Ligure 29. Mandated Pederal Systems Engineering Organization.71 

Ligure 30. CLIOS Representation: Working in Silos - Little Accountability.73 

Ligure 31. Defense Acquisition Accountability.74 


IX 


































THIS PAGE INTENTIONALLY LEET BLANK 


X 



LIST OF TABLES 


Table 1. GAO Status on Selected Major Weapon Systems.7 

Table 2. C arlucci Initiatives.18 

Table 3. Acquisition Improvement Initiatives.20 

Table 4. The Top 15 Military Spender Countries in 2008.21 







THIS PAGE INTENTIONALLY LEET BLANK 



EXECUTIVE SUMMARY 


For decades, the United States’ major defense acquisition system has been under 
scrutiny and undergone much reform. Numerous groups have researched the issues, 
publishing hundreds of reports identifying various problems and solutions. Yet, many 
major weapon systems continue to be well over budget and schedule. 

The U.S. major defense acquisition system is increasing in size, scope, and 
complexity each year. Technology associated with major weapon systems is rapidly 
changing. Customer expectations are rising. Societal concerns, such as workforce and 
economic development, are playing an increasingly bigger role. Politics continue to play 
a part in this system. This system certainly qualifies as a CLIOS system—Complex, 
Large-scale, Interconnected, Open, and Sociotechnical in nature. 

This thesis explored and analyzed the decades of research concerning U.S. major 
weapon systems acquisitions. The research introduced the CLIOS Process developed by 
MIT to analyze and study complex systems. The basic structure of the CLIOS Process is 
three stages encompassing 12 steps. The three stages include (1) Representation; (2) 
Design, Evaluation, and Selection; and (3) Implementation. 

Of the three stages within the CLIOS Process, this research applied the 
Representation stage to the U.S. major defense acquisition system. This exploratory 
approach described the system as a whole, identified subsystems, and described the 
components and links among the subsystems and major actor groups. Despite continuing 
with the CLIOS Process, initial analysis was conducted based on the observations of 
Stage 1. This resulted in two observations: (1) Long-term decisions made on short-term 
information and (2) Independent working leads to lesser accountability. Three strategic 
alternatives were identified: (1) form an Integrated Process Team with Congress, JCIDS 
and the Defense Acquisition System to jointly consider both near and far-term economic 
conditions and security threats before new long-term defense programs were approved 
and funded; (2) form a mandated Federal Systems Engineering organization equally 
accountable to both Congress and DoD to consider economic and threat conditions, 

xiii 



analyze technological maturity, and make independent cost estimates as inputs to new 
defense system acquisition decisions; and (3) create a hybrid of the two previous 
solutions to implement accountability at all levels of the acquisition to all major actor 
groups. Of course, challenges to implementation will exist. However, continuing with 
the CLIOS Process will facilitate in refining the strategic alternatives, as well as 
implementing them. 

Further research should be conducted on the U.S. major defense acquisition 
system using the CLIOS Process. The remaining stages of the CLIOS Process, especially 
the iterative aspects of the process, should be applied to the U.S. major defense 
acquisition system. 



ACKNOWLEDGMENTS 


First, I would like to extend a hearty, Southern, and sincere “Thank you!” to the 
PD-21 faculty and CED3 staff at the Naval Postgraduate School. You all provided an 
excellent learning and development experience for my classmates and me. I must write a 
special note of gratitude and appreciation to my thesis advisor. Dr. John Osmundson (or 
“Dr. O” as Cohort 8 called him), for being the scholar, teacher, and mentor that he is. 
Although many of us heard of him before we arrived in Monterey, Dr. O immediately 
gained the attention and respect of Cohort 8 in that first week. He sustained that 
reputation throughout two years of our learning. Thank you. Dr. Osmundson! You are 
highly esteemed and beloved by your students. 

I thank my company, Northrop Grumman Shipbuilding—Gulf Coast (NGSB- 
GC), and, especially, the Director of Systems Engineering, Robert “Cola” Rifley, for 
allowing me this outstanding learning opportunity and for providing me with an 
assiduous and reflective Second Reader, Charles Wilson. Charles is a graduate of the 
Product Development 21 program and was a great encouragement to me during my 
tenure with NPS. He is also an exceptional manager in the Systems Engineering 
department at NGSB-GC. I am very fortunate to have worked with him on this thesis. 

I want to thank my classmates in Cohort 8. We have shared many experiences 
these past two years. Thank you for the memories. 


Brian—the love of my life! 1 dedicate this to you. “Thanks” is not enough. 


XV 



THIS PAGE INTENTIONALLY LEET BLANK 


XVI 



I. 


INTRODUCTION 


A. BACKGROUND 

For more than a half century, issues surrounding the United States’ major 
weapons system acquisition have been scrutinized and discussed. Watchdog agencies 
have researched the issues and made recommendations, publishing hundreds of reports 
identifying numerous problems and solutions. While improvements and reforms have 
been made, many major weapon systems continue to suffer from requirements creep, 
budget and schedule overruns, performance issues, and much more. According to an 
April 2009 report published by the United States Government Accountability Office on 
major weapon system acquisition, the Department of Defense’s 2008 portfolio of major 
defense acquisition programs is already showing signs of trouble. Total research and 
development costs are on average 42 percent higher than the first estimate. Total 
acquisition costs are 25 percent higher than the first estimate. The average delay in 
delivering initial capabilities is 22 months (GAO, 2009). 

The system for building and acquiring U.S. major weapon systems can be 
classified as a “CLIOS” system—Complex, Large-scale, Interconnected, Open, and 
Sociotechnical in nature (Sussman, 2007). This system of acquiring major weapons 
proves to be more complex and dynamic than ever imagined: 

. There is rapidly changing technology. 

• The time between concept design to delivery can take decades. 

. There are social, political, and economic factors intertwined in the 
process. 

. The end user is not the customer. 

• The government has a need to retain specialized skills, yet has a limited 
number of programs on which to bid and “keep” defense contractors 
active while still meeting requirements. 
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• The U.S. defense industrial base is exclusively private-owned and thus 
governed by independent business decisions. 

. The “military industrial complex” is both a burden and necessity to the 
U.S. government. 

The U.S. major defense acquisition system is a very complex system with sub¬ 
systems and many internal and external influences. Much has been written and 
documented on this process. Repeatedly, groups have highlighted the immediate need of 
meeting planned schedules and budgets. Yet, there is little if any “improvement” on the 
issues that have plagued this industry and process for decades. The focal point of past 
research and publications concentrated on immediate solutions. Although solutions are 
necessary, the same issues and recommendations are given year after year. 

In order to effectively address issues within this giant, complex system, it first 
must be treated as such. The U.S. major defense acquisition is increasing in size, scope, 
and complexity each year. The technology associated with major weapon systems is 
rapidly changing. Customer expectations are rising. Societal concerns such as workforce 
and economic development are playing an increasingly larger role. Politics continue to 
play a part in this system. The U.S. major defense acquisition system certainly qualifies 
as a CLIOS system—Complex, Large-scale, Interconnected, Open, and Sociotechnical in 
nature. 

B. PURPOSE AND BENEFITS OF STUDY 

The purpose of this study is to gain a better understanding of the system of 
acquiring major weapon systems as a whole, identify the emergent behavior of this 
system, find possible points of leverage for improvements to the system, and begin 
applying the CLIOS framework for a different approach and understanding of the system. 

The goal is twofold: exploratory and descriptive. The study will explore previous 
research over the past decades concerning major weapon systems acquisition. Many of 
the findings outline the immediate needs of programs’ ability to meet the set schedule and 
budget. The purpose of exploring the literature is to better understand how the system 
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has behaved in the past, understand the areas of concern, and identify the pieces and 
players within the system. This study will offer an overview of significant research and 
literature published about major weapon systems acquisition. 

This study will also describe the major defense acquisition system in order to 
ensure a holistic view is captured. Architecting the system will aid in the understanding 
of all of the systems at play within the major defense acquisition system as well as the 
interfaces. 

C. RESEARCH QUESTIONS 

This thesis will seek to answer the following research questions: 

What are the issues identified in previous literature concerning U.S. major 
weapon systems acquisition? 

How does the United States’ system of acquiring major weapons compare to other 
top military spending nations? 

What are the pieces, and/or who are the players, within the major defense 
acquisition system? 

What other areas of the major defense acquisition system should be identified and 
researched when applying the CLIOS process? 

Will the CLIOS process provide more understanding of the issues and enable 
solutions to the U.S major defense acquisition process that have not been previously 
reviewed or suggested? 

D. METHODOLOGY 

This exploratory and descriptive research is a case study of the major defense 
acquisition system. In order to effectively address issues within this giant, complex 
system, it first must be treated as such. The U.S. major defense acquisition process is 
only increasing in size, scope, and complexity with each year and certainly qualifies as a 
CLIOS system—Complex, Large-scale, Interconnected, Open, and Socio-technical in 


nature. 
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Chapter II will provide a comparison of the U.S. defense acquisition system to 
acquisition systems of other top military spending countries. It will end with a review of 
the major defense acquisition literature. This literature review will focus on the critical 
points of past and current knowledge of the issues surrounding the U.S. system of major 
weapon systems acquisition. This review will take a step back, explore, and analyze the 
research and publications, which identified issues and solutions towards U.S. major 
weapon systems acquisitions. 

This exploratory approach will seek to identify gaps in the understanding of the 
system as a whole using the CLIOS Process as a framework. The CLIOS Process was 
created by Joseph Sussman, a professor in the Engineering Systems Division Program at 
the Massachusetts Institute of Technology (MIT), in order to “describe, understand, 
study, and, ultimately, to improve the performance of a...system...with wide-ranging 
economic, social, political, and environmental impacts” (Sussman, 2007, p. 6). This 
process will be introduced and reviewed in Chapter III of this study. 

Chapter IV will present a partial CLIOS representation of the major defense 
acquisition system, which is an approach to architecting the system. This allows for 
describing the system, identifying subsystems, and describing components and links. 
Finally, Chapter V will conclude the study with key points and recommendations for 
improvements and further research on the system of acquiring major weapon systems. 
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II. REVIEW OE LITERATURE 


A. INTRODUCTION 

“It is clear that the problems being encountered today are similar to those of the 
past.” (GAO, 1988) 

Issues of increased costs, schedule delays, and degradation of performance 
requirements have plagued the United States’ major weapons acquisition system for more 
than half a century. Scholars, researchers, and practitioners alike have studied the 
acquisition of major weapon systems. The pure quantity of reports generated makes it 
infeasible to review all of them. While a plethora of literature associated with this subject 
exists, only key reports have been chosen to provide a broad understanding of the issues, 
reasons and recommendations, and change initiatives provided by these researchers. 

B. ISSSUES SURROUNDING MAJOR WEAPON SYSTEMS ACQUISITION 

Marshall and Meckling (1959) conducted research on 22 major weapon systems 
to better predict the success of development and production. They identified three factors 
impeding the success of new programs: cost, time, and performance. Marshall and 
Meckling discovered cost increases of 200-300%. The actual costs for missiles were, on 
average, 5.1 times greater than the original estimates. On average, the actual costs for 
bombers were 3.1 times greater than the original estimates. Concerning time, Marshall 
and Meckling found data for only ten of the 22 systems they researched, where time 
extensions of 33-50% were the norm for newer weapon systems acquisitions. Of those 
ten systems, the average “slippage” from original estimate to actual availability was two 
years. Although performance was more difficult to quantify (such as the use of dollars 
and years for the previous two factors), Marshall and Meckling found that most of the 22 
systems “fell short of performance expectations” (p. 21). 

GAO (1969, November) reported on the Army’s purchase of Sheridan weapon 
systems, M60A1E1 tank turrets, and M60A1E2 tanks. The tanks did not meet 
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performance expectations nor schedule deadlines, with time and performance being two 
of the factors identified in Marshall and Meckling’s research. Basically, the tanks were 
rapidly produced before adequate ammunition was created and tested. However, the 
acquisition of tanks continued, ultimately forcing them to be stored unused. 

In 1969, Congress asked GAO to periodically report on the status of various 
major weapon systems. Fifty-seven programs were selected for the status on the 
acquisition of major weapon systems in February 1970 (GAO, 1970). However, only 38 
had sufficient data for comparison. Initial cost estimates were significantly misjudged 
with estimates of program completion 50% higher than original estimates. Along with 
cost, performance was an issue on many of the programs. A large number of weapon 
systems had considerable variance in performance from the original estimate to the 
subsequent estimates before production. GAO also reported schedule delays on most 
programs ranging from six months to three years. 

In June 1971, Perry, Smith, Harman, and Henrichsen published the findings of a 
two-year study on 24 major systems developed during the 1960s. The focus of these 
Rand researchers was the three factors identified by Marshall and Meckling: cost, 
schedule, and performance. Data for the three factors on all 24 systems was not 
available. However, Perry et al. were able to gather significant data as well as research 
causes and possible solutions. The cost data revealed an average increase of 40%, while 
schedule delays averaged 15% (Perry et al., 1971). 

The GAO released a comprehensive review of selected Air Force, Navy, and 
Army major weapon systems for the 1971 status report (GAO, 1971, March). Estimated 
costs for 61 systems increased by $33.4 billion. In the 1972 status, GAO reviewed 47 
other systems that had a net increase of only $2,287.6 million during 1971 (GAO, 1972). 
Between June 30, 1972 and December 31, 1972, a selection of 45 major weapon systems 
increased in cost by $585.7 million (GAO, 1973). For six months during 1973, there was 
a net increase of $7,016.1 mi llion on the 55 major weapon systems under review (GAO, 
1974). The 50 major weapon systems selected in 1974 saw a net increase of $13,242.5 
million (GAO, 1975). Of those 50 major weapon systems, 22 were either 12 months or 

more behind schedule. Surprisingly, the GAO status report published in 1976 showed a 
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net decrease of $2,355.9 million for the 47 major weapon systems analyzed January 1, 
1975 to June 30, 1975 (GAO, 1976). GAO continued to release reports concerning 
increased costs on selected major weapon systems throughout the 1970s. See Table 1. 


Table 1. GAO Status on Selected Major Weapon Systems 


GAO Status on Selected Major Weapon Systems 


Net Increase / 


# of 


Decrease from 

Year 

Weapn 

Year of Data 

Original Estimate 

Published 

systems 

1970 

$33.4 billion 

Kg 

61 

1971 

$2.3 billion 

mm 

47 

1972 

$0.59 billion 

mm 

45 

1973 

$7.0 billion 

1974 

55 

1974 

$13.2 billion 

1975 

50 

1975 

($2.4 billion) 

1976 

47 

1976 

$14.4 billion 

1977 

52 

1977 

(no data) 



1978 

$29 billion 

1979 

60 

1979 

$97.0 billion 

1979 

58 


In his statement to congress in 1979, Jerome H. Stolarow, the Director of 
Procurement and Systems Acquisition, spoke to the lingering problem of cost growth 
saying, “Despite the level of anxiety, the publicity, and the many learned studies, the 
problem persists” (GAO, 1979, June, p. 2). 

Certainly cost growth received much attention in the 1970s. However, after 
thoroughly reviewing 21 major weapon systems, GAO released a report in 1980, which 
identified and grouped issues into 17 categories: (1) operational or performance 
limitations, (2) survivability or vulnerability, (3) availability, (4) operational 
requirements, (5) reliability, (6) force mix requirements, (7) force capabilities, (8) 
operational utility, (9) affordability, (10) data reporting incomplete, (11) program 
concurrency, (12) adequacy of testing, (13) cost effectiveness, (14) program 
management, (15) development strategy, (16) system urgency, and (17) technical risk or 
problems. These 17 categories fell into a broader category of either mission effectiveness 
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or program acquisition. Of the 75 issues found, 59% fell into the mission effectiveness 
category and 41% in the program acquisition (GAO, 1980). 

A similar report was again issued in 1982. GAO selected 24 major weapon 
systems to analyze and found 71 issues that were grouped into 15 categories falling into 
one of two broad areas identified (GAO, 1982). The issues were identified as either a 
system effectiveness issue or a program acquisition issue. System effectiveness 
accounted for 37% of the issues, and program acquisition accounted for 63% of the 
issues. 

In a statement before the Senate in 1984, Charles Bowsher, comptroller general of 
the United States, validated the trends in both cost growth and extended schedules over 
the past decade among major weapon systems and expressed concern at this continued 
problem, seemingly impossible to control. Bowsher emphasized the current seriousness 
of cost growth on major weapon systems despite the recognition it had received for 
decades (GAO, 1984). Four years later, GAO (1988) released another report 
underscoring the continued and recurring problems with cost growth, schedule delays, 
and deficiencies in performance for major weapon system acquisitions over the past 18 
years. 

Despite the reviews and publicity in the previous decades, major weapon systems 
acquisitions continued to have problems throughout the 1990s. In a report released 
December 1992, GAO listed the major persistent issues recurring from program to 
program, year to year: (1) Systems being acquired that may not be the most cost- 
effective solution to the mission need; (2) Overly optimistic cost and schedule estimates 
leading to program instability and cost increases; (3) Programs that cannot be executed as 
planned with available funds; (4) Program acquisition strategies that are unreasonable or 
risky at best; (5) Too much being spent before a program is shown to be suitable for 
production and fielding; and (6) Individuals seeking to improperly influence the outcome 
of the contracting process (GAO, 1992, p. 8). 

A 1998 report released by GAO highlighted DoD’s practice of allowing 
technology development to continue into product development rather than having known 
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technology when production begins (GAO, 1998). Programs continued with much less 
knowledge and, thus, much higher risks. This uncertainty created production problems, 
increased costs, and schedule delays. Programs typically continued without the needed 
technology developed. In fact, many programs overpromised the technical performance 
without true knowledge of the undeveloped systems. These unknowns were not 
classified as high risks, and therefore, did not receive the needed attention in earlier 
stages of development (GAO, 1998). 

By 2005, GAO (now the United States Government Accountability Office) had a 
portfolio of major weapon systems valued at $1.3 trillion (GAO, 2005). In her testimony 
before the Senate, Katherine Schinasi, Managing Director, Acquisition and Sourcing 
Management, repeated the issues of cost growth, schedule delays, quantity reduction, and 
reduced performance (GAO, 2005). Schinasi reported the acquisition process was not 
responsive to current needs and the United States was on an “unsustainable fiscal path” 
that “will gradually erode...our economy, our standard of living, and ultimately our 
national security” (GAO, 2005, p. 3). GAO’s major weapon systems reviews revealed 
that programs with demonstrated or mature advanced technology had costs increases of 
less than one percent per unit cost. Whereas, those programs that began development 
with unproven technology had increased costs of approximately 21% over the first full 
estimate (GAO, 2005). 

At the end of 2005, the Assessment Panel of the Defense Acquisition 
Performance Assessment Project released a report identifying over 42 issue areas within 
major weapon systems acquisition (DAPAP, 2005). Fifteen issue areas were stressed as 
key, going beyond cost, schedule, and performance: program structure, acquisition 
strategy, complex acquisition system, requirements process, joint requirements 
development, need for leadership, process discipline, oversight, requirements allocation, 
program management (PM) expertise, acquisition career path, industry motivation and 
behavior, and the Planning, Programming, Budgeting, and Execution (PPBE) process. 

Again, in 2006, GAO issued a report detailing how continued recommendations 
and changes in policy have not rendered different results within major weapon systems 
acquisition (GAO, 2006). 
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C. REASONS AND RECOMMENDATIONS 

Marshall and Meckling’s (1959) conclusions were (1) early estimates were 
inaccurate, partly due to overoptimism and (2) estimates became more accurate as the 
system progressed. From their research, Marshall and Meckling discovered the penalties 
for underestimating cost and times were minimal compared to the opportunity for 
contractors to obtain the contract. Also, there was much uncertainty surrounding 
technological advances, hence the estimates becoming more accurate as the systems 
progressed. Marshall and Meckling (1959) noted the inaccuracies in cost, availability, 
and performance greatly affected military and procurement decisions. 

In 1969, Elmer Staats, Comptroller General of the United States, gave a statement 
before the Subcommittee on Economy in Government, Joint Economic Committee on the 
Military Budget and National Economic Priorities (GAO, 1969, June). In his statement, 
Mr. Staats confirmed there was no standard procedure for contractors to adequately 
identify problems on major weapons programs concerning cost, schedule, and technical 
performance. Mr. Staats continued to confirm one of the greatest problems with major 
weapon systems acquisition was understanding how changes made to contracts impacted 
cost, schedule, and system performance. 

The United States General Accounting Office (GAO) released a report to 
congress in November of 1969 stating improved management of the Sheridan weapon 
systems, M60A1E1 tank turrets, and M60A1E2 tanks would have minimized the 
situation (GAO, 1969, November). 

The status on the acquisition of major weapon systems released by GAO in 
Eebruary, 1970 stated causes for the cost growth included (1) beginning acquisition 
before the weapon system had been demonstrated with probable successful development, 
(2) inadequate initial requirements and performance specifications, and (3) the difficulty 
to anticipate cost growth on major weapon systems that are produced over a long period 
of time (GAO, 1970). 

Cost growth for the 61 systems in the comprehensive report in 1971 was 


attributed to various reasons: difference between planning and development estimates, 
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changes in quantities, engineering changes, revisions to estimates, and inflation (GAO, 
1971, March). Even more specifically, there were certain fundamental problems: (a) 
identifying the need for a weapon system and prioritizing, (b) determining performance 
characteristics and their feasibility, (c) quality suffering in support of cost-effectiveness, 
(d) organizational structure, and (e) cost estimation. 

The GAO (1971, March) recommended four distinct changes for improvement: 
(1) have a standardized procedure for all military branches to request new systems based 
on mission need and to prioritize those systems, (2) institute standards for preparing, 
updating, and utilizing cost-effectiveness studies, (3) give more decision-making 
authority and control of operations to the corresponding military branch, and (4) 
standardize acquisition reports to contain summary of acceptability of the weapon, 
relation to and recognition of other complementary weapons, and current status. 

In Perry et al.’s (1971) findings, the data indicated that meeting performance 
requirements was the main objective for programs. Schedule had a lesser priority than 
performance, and cost growth was allowable to facilitate meeting performance and 
schedule goals (Perry et ah, 1971). Perry et al. (1971) isolated the causes for cost growth 
in three broad categories: technical uncertainty, scope change, and cost estimating error. 
It was obvious those programs with longer durations and higher technological advances 
were the chief offenders of cost growth. However, across the board, technical uncertainty 
affected all three factors of cost, schedule, and performance. Nonetheless, cost growth on 
fifty percent of the programs was attributed to scope change. 

By 1972, it was evident a leading source of cost growth on major weapons 
systems was technical uncertainty. In response to this, GAO conducted a thorough 
examination of the policies and practices of testing and released a report on the 
importance of testing and evaluation in major weapon system programs (GAO, 1972). 
GAO examined 13 major weapons systems and concluded that testing objectives were 
adequate but were offset by inadequate testing plans. Testing and evaluation were not 
completed in time, and although the generated reports were adequate, the value of these 
reports was inadequate due to the lack of planning and actual testing. GAO 
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recommended testing be conducted before major decisions or milestones, gaining control 
of waivers from required testing, and creating concise summary reports for all levels of 
management (GAO, 1972). 


In many of the periodic updates to congress on selected major weapon systems, 
GAO repeatedly inserted the following statement: 

Overly ambitious performance requirements, combined with low initial 
cost predictions and optimistic risk estimates, lead almost inevitably to 
schedule slippages, performance degradations, and cost increases. 
Attempt to keep total program costs from rising lead to reductions in 
planned quantities that, in turn, increase unit cost. (GAO, 1975, p.24) 

The primary reason for a net decrease in the weapon systems chosen in 1975 was 
due to economic factors, primarily inflation (GAO, 1976). However, cost growth 
continued and in his statement to congress, Stolarowe proclaimed, “Cost growth of 
weapon systems is a highly complex and multi-faceted problem involving economics, 
military judgment and politics” (GAO, 1979, June, p. 4). Stolarowe stated the factors 
leading to increased costs were interrelated and impossible to separate. One factor was 
preliminary cost estimates. Stolarowe acknowledged over-optimistic cost estimates 
become more accurate and realistic the further along in the development cycle because 
more is known about the system. Technology was the second factor because high 
technology systems were more complex needing more research, development, design, 
testing, maintenance, support, etc. The third factor was inflation, and the fourth, funding, 
both causing production delays, changes in production rates, and production 
inefficiencies (GAO, 1979, June). 

GAO’s reports to congress in 1982 and 1983 had many broad and specific 
recommendations concerning major weapon system acquisitions (GAO, 1982; GAO, 
1983). First, risk must be minimized and system effectiveness ensured through improved 
testing and limitations on procurement funds until those risks and uncertainties are 
resolved. Second, congress could make better decisions if it had more information on 
cost, schedule, and performance of major weapon systems provided by the Department of 
Defense. Third, requirements should be established in accordance with mission needs 
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and maintained through completion of the program. Fourth, more analyses of alternatives 
must be conducted to ensure the weapon system is the best all around. Fifth, every 
opportunity must be taken to ensure costs are reduced. Finally, a more effective 
management structure must be established to efficiently accomplish program objectives 
(GAO, 1982; GAO, 1983). 

From research conducted by the General Accounting Office, Charles A. Bowsher 
(GAO, 1984) reasoned cost growth could be more controlled if cost estimating 
procedures and reporting of actual costs were improved. Bowsher recognized the 
vagueness and conflicting guidelines for cost estimating. He conveyed numerous reasons 
for cost growth to the Senate: DoD’s implementation of the guidance was weak, cost 
estimates lack definite structure, definition of programs lacks consistency, documentation 
was insufficient, risks were not properly accounted for, inflation was not consistently 
recognized ,or it was used to hide other costs, improvements were needed in the direction 
provided by DoD, DoD estimates were based on inaccurate or optimistic data, program 
cost estimates were forced to conform to the budget, relevant program costs were 
excluded, and cost estimates would be more reliable by using independent cost estimator 
recommendations (GAO, 1984). 

The 1988 GAO report summarized 18 years of systemic causes of cost growth, 
schedule delays, and performance shortfalls. Several factors were highlighted: (1) no 
funding for analyzing alternatives, which forces one design; (2) premature commitment 
to an unproven design; (3) decisions made during program planning limit program 
managers; (4) lack of upper-management commitment to weapons requirements for 
design; (5) insufficient funding; (6) inappropriate external management direction; (7) 
more commitment to the development schedule existed than commitment to correcting 
serious problems; (8) process used by DoD to determine weapon system requirements; 
and (9) general instability with the acquisition process (GAO, 1988). 

In 1992, GAO attributed many of the persistent, systemic problems to “a 
prevailing culture that is dependent on generating and supporting new weapons 
acquisitions” (GAO, 1992, p. 9). This culture created an environment encouraging the 

selling of programs, and GAO suggested changes be made to the system of incentives 
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and opportunities surrounding the acquisition process (GAO, 1992). This was reiterated 
in GAO’s 1997 report where GAO reviewed six weapons in low-rate production (LRIP) 
and 22 weapons in full-rate production (GAO, 1997). DoD tended to buy large quantities 
of untested weapons in LRIP, which caused many decreases in production rates of tested 
and proven weapons in full production. These proven weapons not only experienced 
reduced production rates but also extended schedules, which, in turn, resulted in higher 
costs per unit. The larger investment in untested weapons resulted in increased costs and 
performance risks. They suggested DoD buy limited quantities of weapons during LRIP 
until appropriate operational tests and evaluations were conducted (GAO, 1997). 

GAO (1998) compared commercial practices to DoD practices concerning 
technology development. Unlike leading commercial firms, DoD consistently began or 
continued programs without the needed technology fully developed, as mentioned in the 
numerous, previous reports. GAO observed DoD acquisition had a culture or 
environment that encouraged such practices, which led to high costs and schedule 
increases. Recommendations included (1) having standards applied to individual 
programs concerning timing and quality of production-related knowledge, (2) splitting 
forced a line of text to next page for more text at top of page technology development 
from weapon systems programs for higher knowledge standards, and (3) allowing 
program managers to identify unknowns as high risks early without criticism or loss of 
funding (GAO, 1998). 

In her statement to the Subcommittee on AirLand, Committee on Armed Services, 
U.S. Senate, Schinasi called for immediate change to the acquisition system, pleading 
with decision makers to make the necessary difficult choices and decisions based in 
reality and to stop being complacent to cost growth, schedule slippages, and performance 
shortfalls (GAO, 2005). Schinasi acknowledged DoD’s policy had adopted a knowledge- 
based, evolutionary approach. However, DoD’s practice was the opposite. Schinasi 
recommended truly implementing the knowledge-based, evolutionary approach to 
acquisition to enable “developers to rely more on available resources rather than making 
promises about unproven technologies” (GAO, 2005, p. 6). This policy fostered a more 
manageable environment, since technology would not be brought into product 
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development unless it was first demonstrated to meet requirements. Nonetheless, 
programs continued to start development with immature technologies. 

To emphasize the need for decision makers to begin making the necessary but 
difficult decisions, Schinasi (GAO, 2005) outlined the current, undisciplined progression 
to product development: 

. DoD’s requirements create more demand for programs than resources. 

. DoD approves too many highly complex and interdependent programs. 

. After a program is approved, DoD adds more requirements, increasing 
costs and risks. 

. Once programs start, the budgeting process exacerbates the problems. 

. Programs are funded annually and department wide. 

. Priorities have not been established. 

• Competition for funding increases. 

. Success is viewed by obtaining next incremental funding, not by 

delivering capabilities. 

• Bad news about programs is suppressed and testing can be skipped to 
lower costs, so as not to lose funding. 

• Senior officials make across-the-board cuts rather than make hard 
decisions about individual programs. 

. Therefore, the acquisition system fosters a culture of beginning programs 
with too many unknowns, risks, and, eventually, shortfalls and higher 
costs (GAO, 2005). 

Schinasi (GAO, 2005) gave specific recommendations for improvement: (a) 
constrain requirements; (b) proved definitive business cases for investments; (c) enable 
science and technology organizations to take the responsibility for technology; (d) 
develop a workforce to manage requirements, source selection, and knowledge-based 
acquisitions; and, finally, (e) enforce knowledge capture and use by decision makers 
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before investing or moving forward with programs. These recommendations demanded a 
disciplined process for requirements and funding (GAO, 2005). 

The 2006 report issued by GAO found DoD still skipping important steps in the 
knowledge-based process and continuing to chase “revolutionary leaps in capability” 
before gathering pertinent knowledge and before technology had matured enough. In 
fact, 80% of the programs reviewed for this report had not followed the knowledge-based 
process before committing to system development (GAO, 2006). 

D. INITIATIVES TO IMPROVE ACQUISITION OF MAJOR WEAPON 
SYSTEMS 

The perennial procurement and contracting cycle—going back many 
decades—of adding layer upon layer of cost and complexity onto fewer 
and fewer platforms that take longer and longer to build must come to an 
end. There is broad agreement on the need for acquisition and contracting 
reform in the Department of Defense. There have been enough studies. 

Enough hand-wringing. Enough rhetoric. Now is the time for action. 

United States Secretary of Defense, Robert Gates, April 6, 2009 

The hundreds of reports released by GAO, the RAND Corporation, and other 
researchers did not go unnoticed. Over the past 50 years, the United States has 
implemented numerous initiatives for improvement of the acquisition system. Over the 
past decades, the fundamental nature of major weapon systems acquisition has changed 
significantly. Unfortunately, as proven in the literature above, even with perpetual 
reports and committees or initiatives, programs continued the cycle of underperforming, 
extending the schedule, and increasing the costs. 

Needed improvements were recognized as early as 1949 with the formation of the 
first Hoover Com mi ssion. The first Hoover Commission, also known as the Commission 
on the Organization of the Executive Branch of Government, did not exactly examine the 
acquisition system as a whole in 1949. However, it did review the role of the Secretary 
of Defense within the process and the structure of the military (McKinney et ah, 1994). 
While the second Hoover Com mi ssion in 1955 was more concerned with commodity 
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goods than weapon systems, it did review the acquisition system more thoroughly 
looking to achieve greater efficiency within the procurement system (McKinney et ah, 
1994). 

During the 1960s, the U.S. Secretary of Defense, Robert McNamara, approached 
the U. S. acquisition system much as he did as president of the Ford Motor Company, a 
large American business (McKinney et ah, 1994). McNamara had a bachelor’s degree in 
economics and philosophy from the University of California, Berkeley, and a master’s 
degree from Harvard Graduate School of Business. McNamara aggressively questioned, 
sought alternatives, and stimulated progress. He was an advocate of “one defense 
policy,” not three conflicting policies for each of the military branches (DefenseLink, 
2009). Some of McNamara’s innovations, such as the Total Package Procurement, had 
unintended consequences like budget overruns. However, some programs such as the 
introduction of a campaign to reduce unnecessary duplication within defense 
procurement had positive impacts (McKinney et ah, 1994). 

In 1970, the first independent, government sponsored commission to thoroughly 
examine the weapon systems acquisition process was the Fitzhugh Commission, also 
known as the Blue Ribbon Defense Panel. The Fitzhugh Commission was also the first 
to address the political aspects of the acquisition system by acknowledging Congress’ 
tendency to micromanage the Department of Defense. The Fitzhugh Commission 
recommended management improvements and changes in contracting regulations 
(McKinney et ah, 1994). A year later in 1971, the introduction of DoDD 5000.1 was a 
major breakthrough in the acquisition process. It illustrated three critical transition 
points, which were later referred to as milestones: (1) Program Initiation, (2) Begin Full- 
Scale Development, and (3) Begin Production/Deployment (Acquisition History Project, 
2009). 

Two years after the report issued by the Fitzhugh Commission, 1972, the 
Government Procurement Commission also recommended management improvements 
and changes in contracting regulations. However, what set the Government Procurement 
Commission apart from the Fitzhugh Commission was the recommendation that 
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Congress have a better understanding of defense as a whole and a more strategic role in 
defense acquisition (McKinney et ah, 1994). 

The Carlucci Initiatives of 1981 had an aggressive agenda: reduce length of 
acquisition process, increase cost savings, improve readiness, and strengthen the defense 
industrial base (Acquisition History Project, retrieved 2009). After reviewing the entire 
system, 32 initiatives were implemented. See Table 2 for a summary of the Carlucci 
Initiatives. 

Table 2. Carlucci Initiatives 


B 

Initiative 

■ 

Initiative 

1 

Reaffirm Acquisition Management Principles 

17 

Decrease DSARC briefing and data 
requirements 

2 

Increase use of Preplanned Product Improvement 

18 

Budget for inflation 

3 

Implement mutliyear procurement 

19 

Eorecast business base conditions 

4 

Increase program stability 

20 

Improve source selection process 

5 

Encourage capital investment to enhance 
productivity 

21 

Develop and use standard operation and 
support systems 

6 

Budget to most likely costs 

22 

Provide more appropriate design-to-cost- 
goals 

7 

Use economicial production rates 

23 

Implement acquisition process decisions 

8 

Assure appropriate contract type 

24 

Reduce number of DSARC milestones 

9 

Improve system support and readiness 

25 

Submit MENS with Service POM 

10 

Reduce administrative costs and time 

26 

Revise DSARC membership 

11 

Budget for technological risk 

27 

Retain USDR&E as Defense Acquisition 
Executive 

12 

Provide front-end funding for test hardware 

28 

Raise dollar threshold for DSARC review 

13 

Reduce governmental legislation related to 
acquisition 

29 

Integrate DSARC and PPBS process 

14 

Reduce number of DoD Directives 

30 

Increase PM visibility of support resources 

15 

Enhance funding flexibility 

31 

Improve reliability and support 

16 

Provide contractor incentives to improve 
reliability 

32 

Increase use of competition 


[From Holbrook, 2003, p. 10]. 
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The Reagan administration created the Grace Commission, also known as the 
President’s Private Sector Survey on Cost Control, to examine the entire government, 
including defense acquisition, looking for ways to avoid wasteful public spending. In 
their report of 1983, the Grace Commission criticized the “excessively complex set of 
regulations” surrounding the acquisition process and, much like the Fitzhugh 
Commission, criticized Congress for micromanaging weapons acquisitions that have a 
cost impact for the interference. Later the Packard Commission built on the Grace 
Commission’s call for greater stability in defense programs and budget (McKinney et ah, 
1994). 

Former President Ronald Reagan, as well, chartered the Packard Commission, a 
blue ribbon commission (Acquisition History Project, retrieved 2009). The Commission 
reviewed the management of the acquisition process, the budget process, and the 
legislative oversight. After studying the defense acquisition system, the Packard 
Commission recommended the appointment of a full-time Under Secretary of Defense 
for Acquisition, which was put into policy in 1986 (Acquisition History Project, retrieved 
2009). Furthermore, the Packard Commission called for unity and trust among Congress, 
the Executive Branch, the military, and defense industries (McKinney et ah, 1994). 

Like many other initiatives and reports before it, the Goldwater-Nichols Act of 
1986 mandated revolutionary reform for national defense. It was the Goldwater-Nichols 
Act that actually implemented most of the recommendations by the Packard Commission 
(Jones, 1999). 

At the time of this writing, the latest reform effort was passed into law on May 22, 
2009. The Weapon Systems Acquisition Reform Act of 2009 was a bill proposed to 
improve the organization and procedures of DoD concerning the major weapon systems 
acquisition. This new law created a position within DoD—Director of Cost Assessment 
and Program Evaluation. This new position will provide advice relating to acquisition 
program cost estimation and analysis to the Secretary of Defense and DoD officials 
(Library of Congress, 2009). 
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“Poor execution of the revised acquisition policy is a major cause of DoD’s 
continued problems” (GAO, 2006). See Table 3 for a partial listing of programs executed 
for the purpose of improving acquisition system. 


Table 3. Acquisition Improvement Initiatives 


Year 

Improvement Initiatives 

1949 

First Hoover Commission 

1953 

Rockefeiier Committee 

1955 

Second Hoover Commission 

1961 

McNamara Initiative 

1970 

Fitzhugh Commission / Biue Ribbon Defense Panei 

1971 

DoDD 5000.1 was issued 

1972 

Commission on Government Procurement 

1978 

Defense Science Board Acquisition Cycie Study 

1979 

Defense Resources Management Study 

1981 

Cariucci Initiatives 

1983 

Grace Commission / President's Private Sector Survey on Cost Controis 

1986 

Packard Commission / Presdident's Biue Ribbon Defense Commission 

1986 

Goidwater-Nichois Act 

1989 

Defense Management Review 

1994 

Process Action Team on Oversight and Review 

2009 

Weapon Systems Acquisition Reform Act 


E. TOP MILITARY SPENDING COUNTRIES 

The United States is the largest consumer of military goods in the world. 
According to the Stockholm International Peace Research Institute (SIPRI) (2008)—a 
Swedish-based independent research institute focusing on international security, arms 
control, and disarmament—the United States’ military acquisition expenditures comprise 
41.5% of the world’s total military expenses. The top five military spending countries, 
which include the U.S.’s 41.5%, make up 60% of the world market share. See Table 4 for 
details on the top 15 military spending countries as of 2008. 
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Table 4. The Top 15 Military Spender Countries in 2008 


Rank 

Country 

Spending 
($ b.) 

World 
Share (%) 

Spending 
per Capita 
($) 

Share of 

GDP, 2007 

{%T 


1 

USA 

607 

41.5 

1967 

4 

66.5 

2 

China 

[84.9] 

[5.8] 

[63] 

[2.0] 

194 

3 

France 

65.7 

4.5 


2.3 

3.5 

4 

UK 

65.3 

4.5 


2.4 

20.7 

5 

Russia 

[58.6] 

[4.0] 

[413] 

[3.5] 

173 

Sub-total top 5 

882 

60 




6 

Germany 

46.8 

3.2 

568 

1.3 

-11.0 

7 

Japan 

46.3 

3.2 

361 

0.9 

-1.7 

8 

Italy 

40.6 

2.8 

689 

1.8 

0.4 

9 

Saudi Arabia” 

38.2 

2.6 

1511 

9.3 

81.5 

10 

India 

30.0 

2.1 

25 

2.5 

44.1 

Sub-total top 10 

1084 

74 




■■ 

South Korea 

24.2 

1.7 

501 

2.7 

51.5 


Brazil 

23.3 

1.6 

120 

1.5 

29.9 

19 

Canada 

19.3 

1.3 

581 

1.2 

3.4 

19 

Spain 

19.2 

1.3 

430 

1.2 

37.7 


Australia 

18.4 

1.3 

876 

1.9 

38.6 

Sub-total top 15 

1188 

81 




World 


1464 

100 

217 

2.4 

44.7 


[ ] = estimated figure; GDP = gross domestic product. 

^ The figures for national military expenditure as a share of GDP are for 2007, the most recent year for which 
GDP daa is available. 

“ The figures for Saudi Arabia include expenditure for public order and safety and might be slight 
overestimates. 


Due to public access and/or language barriers, it was not possible to gather 
information on all of the above countries’ defense acquisition systems. However, basic 
information about defense acquisition was found on seven of the top 15 countries: 
United States, France, Germany, United Kingdom, Australia, Japan, and South Korea. 

1. The United States’ Defense Acquisition System 

According to the CIA world Factbook, the United States (U.S.) has the largest 


economy in the world. The U. S. has a constitution-based federal republic government 

with a strong democratic tradition. It has a population of 307,212,123. The U.S. Armed 

21 














































Forces has four branches: U.S. Army, U.S. Navy including the Marine Corps., U.S. Air 
Force, and U.S. Coast Guard (CIA World Factbook, 2009). 


The U.S. political system has a series of checks and balances concerning DoD 
weapon systems acquisition. Congress plays a significant role in oversight and controls 
the “power of the purse” (Kausal, 1999, p.4-10). The DoD requests specific weapon 
systems and justifies the need. From this, the President submits a budget to congress. 
After deliberation and modifications, it may be passed, and then the budget is sent back 
to the President for his signature or veto. This process typically takes eight months. See 
Figure 1 for the typical flow of budget for the U.S (Kausal, 1999). 



Figure 1. Typical Flow of Budget [From: Kausal, 1999]. 


The DoD has three decision-making support systems for defense acquisition: Joint 
Capabilities Integration and Development System (JCIDS); Planning, Programming, 
Budgeting, and Execution (PPBE) System; and Acquisition Management System (DoD 
5000.02). Eigure 2 depicts these three systems enclosed within the “Big ‘A’ Acquisition 
Process.” 
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THE ACQUISITION SYSTEM 


BIG “A" 

ACQUISITION PROCESS 



EFFECTIVE INTERACTION 
ESSENTIAL FOR SUCCESS 


LITTLE "i" 

ACQUISITION PROCESS 
(MOST PREVIOUS EFFORTS 
FOCUSED HERE) 


Figure 2. The Acquisition System [From: Defense Acquisition Performance Assessment 

Report, 2006, p. 4]. 


The Joint Capabilities Integration and Development System determines future 
military needs. Each of the military services is responsible for identifying gaps within its 
own mission requirements, determining new mission needs, and generating requirements 
to meet those needs (DAU, 2009). All major weapon systems acquisitions, also known 
as Acquisition Category (ACAT) I programs, are founded on validated mission needs 
where the requirement is approved by the Joint Chiefs of Staff. The acquisition system 
is managed by the Under Secretary of Defense (Acquisition and Technology) (Kausal, 
1999). 

The Planning, Programming, Budgeting, and Execution System helps with 
creating DoD’s piece of the President’s Budget (DAU, 2009). Three of the phases— 
planning, programming, and budgeting—usually take three years. Planning is a six- 
month process and begins two years before the fiscal year it will be requested. During 
Programming, the military branches issue their Program Objective Memoranda (POM), 
stating their requirements to the Joint Chiefs of Staff (JCS). The military branches, the 
JCS, and the Office of Secretary of Defense discuss over a period of time and agree to a 
number of programs. The Secretary of Defense then issues a Program Decision 
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Memorandum (PDM) (Kausal, 1999). Finally, in the Budgeting phase, the DoD budget 
is finalized and submitted to be included in the President’s budget (Kausal, 1999). 

The Acquisition Management System (DoD 5000.02) contains the policies and 
procedures guiding DoD and consists of five phases, three milestones, and three decision 
points. Once the Materiel Development Decision has been made, the Milestone Decision 
Authority will allow entrance into any point in the system as seen in Figure 3 as long as 
phase-specific entrance criteria and statutory requirements have been met (DoDI 5000.02, 
2008). 


User Needs 


Technology Opportunities & Resources 


The Materiel Development Decision precedes 
entry into any phase of the acquisition 
management system 

Entrance criteria met before entering phase 

Evolutionary Acquisition or Single Step to 
Full Capability 


^ 


(Program 

Initiation) 




IOC 


FOC 


Materiel 

Solution 

Analysis 

Technology 

Development 

Engineering and 
Manufacturing 
Development 

Production & 
Deployment 

Operations & 
Support 

^ M«teri«l 
j Development 
^ Decision 


.•*\Post- /\Post- 

V pdra\/cdra 

LRIP/IOT&E A D^^ision 

V Review 



Pre-Systems Acquisition/\ 


A 


Systems Acquisition 


Sustainment 


Decision Point /\ - Milestone Review Decision Point if PDR is not conducted before Milestone B 


Figure 3. Defense Management Acquisition System [From: DoDI 5000.02, 2008]. 


The United States has more than 80 major weapon systems currently in 
development (DoD FY2009 Budget Request Summary Justification). The acquisition of 
major weapon systems represents 20 percent of the Department of Defense’s annual 
spending on purchases (Hearing on the Reform of Major Weapon Systems Acquisition, 
Defense & Security News—ByU.S.Government on May 5, 2009). 

2. France’s Defense Acquisition System 

According to the CIA World Factbook (2009), France has the eighth largest 

economy in the world. France’s government is a republic, and the country has a 

population of 64,057,792. France has four military branches: Army, Navy, Air Force, 

and National Gendarmerie (CIA World Factbook, 2009). 
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Parliament creates the laws concerning the defense budget, military policy, and 
defense organizational structure (Kausal, 1999). The Ministry of Defense implements 
these policies with the aid of the Joint Armed Forces Chief of Staff, the Delegation 
General for Armaments (DGA), and the Secretariat General for Administration (Kausal, 
1999). See Figure 4 for more detail. 



Figure 4. Organization of National Defense [From: Kausal, 1999]. 


The DGA was created in 1961 and is responsible for France’s weapon systems 
acquisitions (Kausal, 1999). Like the U.S., France has had problems of schedule delays 
and cost overruns with its complex, major weapons acquisitions. Also like the U.S., 
France has undergone acquisition reform to address those issues and to allow for greater 
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flexibility in the process. The new acquisition process consists of stages and phases. The 
four stages are preparation, design, realization, and utilization (Kausal, 1999) (see Figure 
5). 



Figures. New Acquisition Process. 

During the Preparation Stage, operational requirements are outlined while current 
assets are evaluated. Recommendations are given to fill the identified gaps such as 
developing advanced technology or buying off-the-shelf solutions. Throughout this 
preliminary stage, many technical and financial studies are made, and research and 
development programs are initiated. By the end, operational requirements have been 
refined and outlined, resources have been decided, and preliminary budgets have been set 
(Kausal, 1999). 

At the Design Stage, a decision is made, so the Feasibility and Definition Phases 
can begin. A program director and a program officer are appointed when a joint decision 
has been made to move forward. At this point, 80% of the costs of the program will be 
determined, leaving 20% for disposal during the development stage. The defense 
industry submits proposals. Once operational and technical specifications are defined, 
the defense industry will compete for the contract (in most cases), and an industry partner 
is chosen (Kausal, 1999). 
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The Realization (Development/Industrialization and Production) Stage involves a 
formal schedule, performance requirements, and a set budget. A very rigorous procedure 
is put into place for the development and development tests, ensuring no duplication 
between the contractor and DGA. The government firmly commits to a multi-year 
contract with the contractors. This gives the contractors stability needed to invest in 
production at lower cost (Kausal, 1999). 

Finally, the Utilization Stage begins after a weapon system has been formally put 
into operational use. This, of course, is preceded by much testing and acceptance by the 
DGA. During the course of the system’s lifetime, it is assessed for needed updates, 
ultimately resulting in engineering changes (Kausal, 1999). 

3. Germany’s Defense Acquisition System 

According to the CIA World Factbook, Germany has the fifth largest economy in 
the world (in purchasing power parity), only behind the U.S., China, Japan, and India. 
Germany is a federal republic with a population of 82,329,758. Germany’s military, the 
Federal Armed Forces, has five separate branches: Army, Navy, Air Force, Joint Support 
Services, and Central Medical Service (CIA World Factbook, 2009). 

Germany has a Federal President, who is Head of State, as well as a Chancellor, 
who is Head of the German Federal Government (Kausal, 1999). The Chancellor plays 
two roles in defense: (1) sets policy for military issues and (2) in times of war, becomes 
the power of command. Each of the individual military branches develops requirements 
and submits them to the Director General of Armaments (DGA) for review and approval. 
The DGA is also responsible for providing targets for cost, schedule, and performance. 
The Federal Office of Military Technology, which falls under the Federal Ministry of 
Defense (FMOD), is a civilian-run operation and is in charge of weapon systems 
acquisitions, among other responsibilities. However, all budgets for major weapon 
systems acquisition must be approved by Parliament (Kausal, 1999). 

Major weapon systems are developed in phases: Pre-Phase, Definition Phase, 
Development Phase, Procurement Phase, and In-Service Phase. See Figure 6 for more 
detail. 
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Figure 6. Weapon Systems Development Process [From: Kausal, 1999, p. 2-21]. 


At the end of each phase and before entering a new phase, a decision and approval 
are required before the program can continue (Kausal, 1999). The Pre-Phase verifies the 
military mission needs. During this phase, military and industry work together to 
evaluate needs, solutions, and alternatives. Once a solution has been finalized, submitted 
for review, and approved. Definition Phase begins. Definition Phase finalizes weapon 
specifications and establishes the project manager and team working groups. It is only 
finished when the Development Baseline is approved. After this approval, the 
Development Phase begins. At some point in this phase, the prime contractor is chosen 
through a fair and open competition, with the development contract defining the 
responsibilities of the contractor. Operational testing is conducted during this phase, and 
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it concludes with the approval of the document, Approval for Production. The 
Procurement Phase includes series production and the selection of a contractor for the 
entire procurement phase, again this is accomplished through fair and open competition. 
At the end of the Procurement Phase, the weapon system is handed over to the military, 
which begins the In-Service Phase (Kausal, 1999). 

4. The United Kingdom’s Defense Acquisition System 

The United Kingdom (U.K.) has the sixth largest economy, according to the CIA 
World Factbook (2009). The U.K.’s government is a constitutional monarchy and 
Commonwealth realm. The population is 61,113,205. The U.K.’s military branches are 
the Army, Royal Navy, and Royal Air Force (CIA World Factbook, 2009). 

The U.K. has both a Monarch and a Prime Minister. The Monarch is the Head of 
State and the Commander in Chief of the Armed Forces. However, she is constitutionally 
bound to take advice from the Prime Minister. It is the House of Commons that formally 
approves funds for weapon systems acquisition. However, the House of Commons 
approves the budget only, it does not have a say on individual programs. Individual 
major weapon systems needs and requirements are determined by the Equipment 
Approvals Committee, who compares costs to operational effectiveness, as well as other 
trade-off analyses. The committee then gives recommendations to the Ministry of 
Defence. Also, the Defence Evaluation and Research Agency conducts research into 
advanced technology and makes recommendations to the Ministry of Defence (Kausal, 
1999). 

The actual acquisition and management of major weapon systems are the 
responsibilities of the Defence Procurement Agency, which is accountable to Parliament. 
It is the Agency’s goal to obtain the best value for money and this is accomplished 
through competitive bids. Eoreign contractors are free to bid on the majority of defense 
acquisitions, as the prime contractor or as subcontractors. Like the U.S., the U.K. has had 
issues in cost, schedule, and performance on major weapon systems acquisition. 
Therefore, after much review, the Labour Government created the Smart Procurement 
Initiative in 1998, which takes a systems approach to acquisitions, improves requirements 

29 



management, strengthens partnerships with industry, controls schedule, commits to 
longer periods for acquisition funding, and implements incremental acquisition (Kausal, 
1999). 

The U.K.’s new acquisition system reduces the number of formal approvals and 
phases. The acquisition system begins with the Concept Phase to identify best 
alternatives for mission needs. Approval for the necessary resources is given at the Initial 
Gate before beginning the Assessment Phase. During the Assessment Phase, a single 
option is chosen from alternatives for demonstration. Also during this phase, technical 
risks are reduced to an acceptable level, requirements are set and linked to mission needs, 
firm life-cycle costs are set, and up to 15% of total project costs can be spent (Kausal, 
1999). 


The Main Gate is the major approval point where commitment to an individual 
project is made. It is at this point that projects are cancelled, if they do not provide an 
acceptable trade-off between cost and performance—best value for money. If approval is 
given, then the program proceeds to the Demonstration Phase where a contractor is 
selected and a contract is signed for development and production. Right through the 
Demonstration Phase, design continues as requirements and performance are met at a 
fixed cost. Models or prototypes are built to demonstrate integration and capabilities. 
The Manufacture Phase starts production of the system. The system will be tested by the 
manufacturer and end-user before it is officially accepted and put into service. See 
Figure 7 for more detail. 



Figure 7. The U.K.’s Defence Procurement Phases [From: Kausal, 1999, p. 3-24]. 
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5. 


Australia’s Defense Acquisition System 


The island, country, and continent—Australia—has the 18* largest economy 
compared to other countries (CIA World Factbook, 2009). Australia’s government is a 
federal parliamentary democracy and a Commonwealth realm. It has a population of 
21,262,641. The Australian Defense Force has four braches: Australian Army, Royal 
Australian Navy, Royal Australian Air Force, and Special Operations Command (CIA 
World Factbook, 2009). 

Australia has the world’s largest coastline to defend, yet it has no identifiable 
threat. Therefore, Australia struggles with how to structure defense as well as how much 
to invest. Australia’s Defence Organization is the combination of the Australian Defence 
Force (ADF) and the Department of Defence. The Chief of Defence Force (CDF) is in 
command of the ADF, and the Secretary of Defence is head of the Department of 
Defence. Both of these positions are accountable to the Minister of Defence and the 
government for the management of Defence. Like many countries, Australia’s defense 
acquisition system has undergone major reform. To gain better understanding and 
control of acquisition, a new structure was created for the Defence Acquisition 
Organisation (DAO) (Kausal, 2000). See Figure 8 for more detail. 
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Figure 8. Defense Acquisition Organizational Structure [From: Kausal, 2000, p. 1-41]. 


Australia’s weapon systems acquisition process has three main phases: Pre¬ 
project Approval Phase, Approval Phase, and Post-Project Approval Phase. The Pre- 
Project Approval Phase deals with capability development, mission needs, program 
definition, analysis of alternatives, acquisition strategy, military priorities, and approval 
for proposed programs. The Post-Project Approval Phase, also known as the 
Implementation Phase, asks for bids from industry, evaluates and selects contractors. 
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manages the acquisition and contract, and oversees the weapon systems’ acceptance into 
operational service. See Figure 9 for more detail on Australia’s acquisition life-cycle. 
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Figure 9. Australia’s Acquisition Life-cycle [From: Kausal, 2000, p. 1-49]. 


Australia’s once publically owned defense industry has been privatized as a 
means to create more efficiency. Foreign competitors are welcomed to bid. However, 
Australia stipulates certain long-term commitments and investments from these firms. 
The defense culture has changed tremendously in Australia. Now, local industry is 
increasingly involved in capability development, capability management, and cost 
estimates (Kausal, 2000). 

6. Japan’s Defense Acquisition System 

Compared to other countries, Japan has the third largest economy with a 
population of 127,078, 679 (CIA World Factbook, 2009). Japan has a parliamentary 
government with a constitutional monarchy. The Japanese Ministry of Defense oversees 
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three military branches: Ground Self-Defense Force, Maritime Self-Defense Force, and 
Air Self-Defense Force (CIA World Factbook, 2009). 

Although Japan’s Prime Minister is the head of government, the executive power 
of the state is held by the Cabinet. The Cabinet is comprised of the Prime Minister and 
other Ministers of State who are appointed and removed by the Prime Minister. Part of 
the Office of the Prime Minister is Japan’s Defense Agency. This is led by the Director 
General, who is the Minister of State for Defense (Kausal, 2000). See Figure 10 for more 
detail on the structure of Japan’s Defense Agency. 



Figure 10. Japan’s Defense Agency [From: Kausal, 2000, p. 2-15]. 


Weapon systems acquisition in Japan is primarily managed by three bureaus: the 
Bureau of Defense Policy, the Bureau of Finance, and the Bureau of Equipment. The 
Bureau of Defense Policy is responsible for designing defense policy and programs, 
conducting trade-off analyses, and defining the military’s operational activities. The 
Bureau of Finance is involved in creating the defense budget and military spending 
priorities. The Bureau of Equipment is responsible for oversight and management of the 
acquisition system (Kausal, 2000). 
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In Japan, mission needs are defined by the individual military branches and 
proposed to the defense department in the form of new equipment needs or changes to 
existing systems. At this point, an Operational Requirements Document is prepared and 
the Research and Development Process begins, which consists of four phases and 
initializes the acquisition process: Concept Phase, Research Phase, Development Phase, 
and Operation Phase. The Concept Phase conducts feasibility studies, defines costs, and 
analyzes technical challenges. The Research Phase identifies technical risks, 
demonstrates and tests technology, and performs sub-system research. The Development 
Phase requires the contractor to design the system, build a prototype, and test for 
performance. If the system performs acceptably, then it is ready for the production 
phase. However, funding must first be secured and commitment gained from the Defense 
Agency and others. The Operation Phase begins with operational tests. Once these are 
successful, approval is given to award contracts for production (Kausal, 2000). See 
Figure 11 for more information on Japan’s acquisition system. 



Figure 11. Japan’s Acquisition System [From: Kausal, 2000, p. 2-26]. 
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7. 


South Korea’s Defense Acquisition System 


South Korea has the 14th largest economy in the world compared to other 
countries (CIA World Factbook, 2009). It is a republic with a population of 48,508,972. 
South Korea’s military has three branches: Republic of Korea Army, Navy, and Air 
Force (CIA World Factbook, 2009). The President is elected to a single, five-year term 
and is the Head of State, Chief Executive, and Commander-in-Chief of the Armed 
Forces. The Ministry of National Defense is one of the President’s responsibilities. It 
was reorganized in May 1999 (Kausal, 2000). See Figure 12 for the organizational 
structure of the Ministry Of National Defense. 



Figure 12. The Republic of Korea’s Ministry of National Defense [From: Kausal, 2000, 

p. 3-4]. 


The Ministry of National Defense has recently reformed its acquisition system, 
introducing a new acquisition development structure in February 2000. Korea’s 
acquisition structure consists of the Defense Acquisition Office, the Planning and 
Management Office, and the Policy Office, all of which fall under the Minister and Vice 
Minister of National Defense. This new development structure was formed to reduce 
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processes and time, facilitate economical acquisition, and establish clear lines of 
communication (Kausal, 2000). See Figure 13 for a more detailed view. 



Figure 13. Korea’s Acquisition Development Structure [From: Kausal, 2000, p. 3-17] 


Requirements for new weapon systems can come from different sources, not just 
the military branches. However, much like the U.S., each service does propose new 
weapon systems through the document. Required Operational Capability. Approval of 
major weapon systems must be obtained by the Joint Chiefs of Staff (JCS). The 
acquisition process consists of five phases: Planning, Programming, Budgeting, 
Executing, and Evaluation. The Planning Phase first assesses the threat and then 
determines the strategy. Requirements are further explored and a Medium-Long Range 
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Force Requirements Proposal is created. This proposal is evaluated by certain members 
of the JCS; and if they agree on the proposal, they prepare the Joint Strategic Objective 
Plan. This is then reviewed by other members of the JCS; and if these members agree, 
they prepare the Mid-Term Defense Plan and the Defense Acquisition Development Plan. 
These two plans define the Programming Phase. During the Budgeting Phase, the 
Defense Budgeting Document is organized. In order to allow more stability within the 
programs, multi-year funding is requested for the entire project. Once this is approved by 
the Minister of National Defense, it is submitted to the President’s Budget Administration 
Office in May. This is reviewed by the cabinet and then sent to the National Assembly 
for review and approval. Budget is usually passed in December. The Execution Phase 
awards contracts for research and development and for procurement. The Evaluation 
Phase analyzes completed programs and provides feedback (Kausal, 2000). See Eigure 
14 for more detail on the Acquisition Process. 
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Eigure 14. Korean Ministry of National Defense Acquisition Process [Prom: Kausal, 

2000, p. 3-27]. 
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III. METHODOLOGY 


A. CLIOS PROCESS 

“When we try to pick out anything by itself, we find it hitched to everything else 
in the universe.” -John Muir 

Complex systems with wide ranging economic, social, and political impacts are 
more common than ever before, yet understanding their true nature and solutions for 
problems surrounding them is less known. A CLIOS system is one that is Complex, 
Large-scale, Interconnected, Open, and Socio-technical (Sussman, 2007). Professors and 
doctoral students in the Engineering Systems Division at MIT developed the CLIOS 
process for just such systems. It is now taught in the class, “Frameworks and Models in 
Engineering Systems.” Engineering systems are ones with a technical aspect to them, not 
just economic, political, or social. The CLIOS framework is an iterative process used to 
study CLIOS systems. It is a systems approach to analyzing and addressing problems by 
representing the entire system in an integrated form. 

Complexity is a key aspect of a CLIOS system (Sussman, 2007). First, structural 
complexity is an attribute of a system with many interconnected parts. Behavioral 
complexity is an attribute of a CLIOS system, e.g., it is difficult to predict system 
behavior. The internal behavior of subsystems and components may be understood, yet 
the relationship among all of these subsystems and components may be difficult to 
predict. Nested complexity occurs when a physical or technical system is embedded and 
interacts with an institutional sphere. The institutional sphere is a system of its own, 
which may experience structural and behavioral complexity. It is this interaction of the 
two systems—physical and institutional—within one bigger system, which creates nested 
complexity. Finally, evaluative complexity refers to the involvement of multiple 
stakeholders with multiple perspectives, expectations, and values (Sussman, 2007). 

In addition to being complex, CLIOS systems are large-scale, interconnected, 
open, and sociotechnical (Sussman, 2007). Large-scale means these systems have a large 
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impact, are long-lived, and usually are large-scale in geographic sense. A CLIOS system 
is interconnected meaning it is interrelated with other sociotechnical systems. Sussman 
(2007) gives the example of the relationship among transportation systems, energy 
systems, and the global climate systems as being interconnected. Above and beyond the 
technical aspect, a CLIOS system is also open, meaning it consists of social, political, and 
economic characteristics. Finally, a CLIOS system is sociotechnical in that it is not solely 
technical, nor is it exclusively social. See Figure 15 below. 
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Figure 15. CLIOS System—Physical Domain with Subsystems Embedded in an 

Institutional Sphere 


B. CLIOS STRUCTURE 

The basic structure of the CLIOS analysis process is three stages encompassing 
12 steps (Sussman, 2007) (See Figure 16 below). The three stages include (1) 
Representation; (2) Design, Evaluation, and Selection; and (3) Implementation. The 
twelve steps make up the process with Steps 1 through 5 falling within the Representation 
stage; Steps 6 through nine in the Design, Evaluation, and Selection stage; and Steps 10- 
12 in the Implementation stage: 


40 



1. Describe CLIOS System: Checklists & Preliminary Goal Identification 

2. Identify Subsystems in Physical Domain & Groups on Institutional Sphere 

3. Populate the Physical Domain & Institutional Sphere 

4. a.) Describe Components and b.) Describe Links 

5. Transition from Descriptive to Prescriptive Treatment of System 

6. Refine CLIOS System Goals & Identify Performance Measures 

7. Identify & Design Strategic Alternatives for System Improvements 

8. Identify Important Areas of Uncertainty 

9. Evaluate Strategic Alternatives & Select “Bundles” 

10. Physical Domain / Subsystems 

11. Institutional Sphere 

12. Evaluate, Monitor, & Adapt Strategic Alternatives for CLIOS System 
(Sussman, 2007) 
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Figure 16. CLIOS Process (From: Sussman, 2007, p. 15) 


The CLIOS Process requires analysts to take a strong systems-thinking approach. 
It provides a framework for understanding systems holistically and more clearly 
(Sussman, 2007). 
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1. Representation Stage 


The purpose of the representation stage is to explore and come to an 
understanding of the structure and behavior of the system being analyzed (Sussman, 
2007). This stage primarily utilizes diagrams to illustrate the system, subsystems, 
institutional sphere, and the interrelationships among them. At this point, there is a 
common understanding of the system, which leads to better understanding the issues and 
goals. 

The representation stage encompasses the first five steps to illustrating and 
interpreting the system. Step 1 requires checklists and preliminary goal identification 
(Sussman, 2007). It is during this step that the CLIOS System is preliminarily bounded. 
The first checklist, the characteristics checklist, may include the following: 

Temporal and geographic scale of the system, (b) the core technologies 
and systems, (c) the natural physical conditions that affect or are affected 
by the system, (d) the key economic and market factors, (e) important 
social or political factors or controversies related to the system and (f) the 
historical development and context of the CLIOS System. (Sussman, 

2007, p. 19) 

The second checklist requires describing opportunities, issues, and challenges 
(Sussman, 2007). These will be used in Stage 2 to determine improvements through 
strategic alternatives. The final checklist establishes preliminary system goals, which 
usually relate to the results from the second checklist (Sussman 2007). See Figure 17 
below. 
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Figure 17. CLIOS System Checklists (From: Sussman, 2007, p. 20) 
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Since the CLIOS framework is an iterative process, system boundaries may 
expand or contract or even be redefined as the analysis progresses (Sussman, 2007). 

Step 2 of the representation stage requires the analyst identify subsystems in the 
physical domain and groups in the institutional sphere (Sussman, 2007). There are three 
main activities in Step 2: (1) defining the major subsystems in the physical domain, (2) 
determining the main actor groups in the institutional sphere, and (3) defining the 
interrelationships between these on a macro-level. There are various ways to accomplish 
this. One is by using the information gathered in Step 1 and grouping the issues into 
categories. Another is by classifying the subsystems by common functions, technological 
aspects, or relationships to other actor groups in the institutional sphere. For the physical 
domain, the physical system is broken out into subsystems, and the linkages among the 
systems are identified. This may be difficult to develop but valuable due to the better 
understanding of the system structure and behavior. For the institutional sphere, the 
information for the major actor groups can be obtained from the information gathered in 
Step 1 (Sussman, 2007). 

While Step 2 requires identifying subsystems in the physical domain and actor 
groups in the institutional sphere. Step 3 requires populating the physical domain and 
institutional sphere (Sussman, 2007). For the physical domain, early CLIOS diagrams 
are formed to detail each of the subsystems with components and links. There must be a 
balance between adding too much detail and oversimplifying the subsystem and 
components. The institutional sphere is also populated with individual actors within 
major actor groups, along with the links among them. 

Step 4A requires describing these components and actors in the physical domain 

and institutional sphere respectively that were earlier identified and diagramed (Sussman, 

2007). Within the physical domain, the individual components are defined and 

categorized into one of three types of components: regular components, policy levers, or 

common drivers. Regular components are the most common and can refer to complex 

structures such as economic growth or concepts such as congestion. Policy levers are 

components that are influenced by decisions made by the actors in the institutional 

sphere. Common drivers are components that are shared across subsystems. In addition, 
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there can be external factors, which may be a regular component or common driver. To 
differentiate these in the diagram, CLIOS has three shapes and one shading. See Figure 
18. 




external 

factor 


Figure 18. Suggested CLIOS System Diagram Component Shapes (From: Sussman, 

2007, p. 24) 

The actors in the institutional sphere are also described as part of Step 4A. It is 
important to identify characteristics such as their influence over subsystems, their 
relevance to the subsystems, their knowledge and resources, their agenda concerning 
strategic alternatives, and other relevant information (Sussman, 2007). 

Step 4B requires the analyst to describe the links denoting directionality of 
influence, extent of influence, timeframe of influence, and uncertainty (Sussman, 2007). 
The CLIOS process has three classes of links: (1) Class 1: links between components, 
(2) Class 2: links between components and actors, and (3) Class 3: links between actors. 
Figure 19 identifies suggested link shapes for CLIOS diagrams. 
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LINK 

SHAPE 

Class 1 (link between components of physical subsystems) 
Class 3 (link between actors on the institutional sphere) 


Class 2 (links “piojscting’' interactions between the 
institutional sphere and the physical domain) 

-► 

Weak 


Average 

-► 

Strong 


Bi-directional 

A -► 

Positive (increase in component A results in increase in 
component B) 


Negative (increase in component A results in decrease in 
component B) 

-► 


Figure 19. Suggested Link Shapes (From: Sussman, 2007, p. 27). 


The final step in the representation stage is Step 5, which is a transition from 
descriptive to prescriptive treatment of the system. At this point, the general structure of 
the system has been determined, and the behavior of the components, actors, and links 
has been described. Before proceeding to Stages 2 and 3, the analyst should reflect and 
ask questions such as (1) are there strong interactions within or between subsystems or 
actors? (2) which components are influenced by many different actors? and (3) are 
relationships between actors full of conflict or cooperation? Many more questions should 
be asked to better understand the system as a whole before moving to the next stages 
(Sussman, 2007). 

2. Design, Evaluation, and Selection Stage 

Stage 2—Design, Evaluation, and Selection—moves from stage one where initial 
structure and behavior were studied to the next stage where system complexity is 
examined in greater depth to find leverage points for improvement (Sussman, 2007). At 
this stage, strategic alternatives are developed. Where stage one was qualitative, this 
stage is more quantitative using models for comparing strategic alternatives. 
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Stage 2 encompasses Steps 6 through 9 to design, evaluate, and select (Sussman, 
2007). During Step 6, the analyst refines the CLIOS system goals, which were initially 
created in stage one, and identifies performance measures. Performance measures 
identify progress made from the current state of the system to the desired future state as 
determined when refining the goals. Performance measures can be very difficult to 
define and prioritize. 

In Step 7, the analyst identifies and designs strategic alternatives for CLIOS 
system improvement (Sussman, 2007). Once goals and performance measures are 
established, the natural progression is to find areas of improvement. Identifying strategic 
alternatives requires creative thinking and brainstorming. Strategic alternatives are 
physical, policy-driven, or actor-based. A physical strategic alternative may be derived 
after bottom-up systems engineering approach is taken to analyze the physical domain, 
subsystems, and components. A policy-driven strategic alternative may be derived after 
analyzing how various policies influence the system. Finally, an actor-based strategic 
alternative may be derived after evaluating institutional arrangements. 

Step 8 requires flagging important areas of uncertainty (Sussman, 2007). This 
step may be conducted in parallel with Step 7 when identifying strategic alternatives for 
performance improvement. Information gained in Steps I through 6 may provide 
insights into uncertainties. Links with strong interactions may be an area of uncertainty. 
Conflict between actors may be another uncertainty. Common drivers may have strong 
influences and, therefore, be another area of uncertainty. Methods for determining 
uncertainty can be quantitative or qualitative such as risk assessments or scenario 
planning. 

The final step in stage two. Step 9, is to evaluate strategic alternatives and bundles 
(Sussman, 2007). Here the strategic alternatives determined in Step 7 are evaluated using 
models developed in Step 6. Additional models may be necessary. Each of the strategic 
alternatives is evaluated by the impacts it has on the CLIOS system. Trade-off analysis 
may also be used to compare strategic alternatives. It is rare for a single strategic 
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alternative to be chosen and implemented due to the complexity of CLIOS systems. 
Usually, one alternative will not meet the CLIOS system goals. Therefore, strategic 
alternatives may be combined into bundles. 

3. Implementation Stage 

The final CLIOS process stage is implementation (Sussman, 2007). Although 
bundles of strategic alternatives have been chosen, implementation of those alternatives 
may have hurdles of its own. The implementation stage designs a guide for 
implementing the alternatives in Steps 10 through 11. Step 12 is the final step in the 
implementation stage and the CLIOS Process as a whole. 

Step 10 focuses on designing and implementing the plan for the physical domain 
and subsystems (Sussman, 2007). Many questions arise and must be answered during 
this phase before the plan is fully developed. Are there enough resources? Is the time- 
frame for meeting system goals reasonable? How do the strategic alternatives fit with 
each other? How do failures in meeting targets affect the implementation? Which 
actor(s) will implement, monitor, and enforce which strategic alternative? Attention must 
be given to the performance measures and trade-offs. 

Step 11 focuses on designing and implementing a plan for the institutional sphere 
and may be conducted in parallel with Step 10 (Sussman, 2007). Some of the strategic 
alternatives developed in Step 9 may include actor-based changes and must be included 
in the overall implementation plan. The analyst must understand the actor(s), the actor’s 
goals, and the changes that will affect the actor(s). There may be opponents of these 
alternatives with resistance to implementation. 

Step 12 ensures that the implemented alternatives are monitored and evaluated 
both in the short- and long-term (Sussman, 2007). Although the strategic alternatives are 
evaluated in Step 9, it is impossible to know their true impacts on the system as a whole 
until after implementation. It is possible to have unanticipated and/or indirect side 
effects. During Stepl2, outcomes are observed, and necessary modifications are made. 
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C. CONCLUSION 

The CLIOS Process is iterative in nature. Goals and performance measures set in 
greater depth in Step 6 may require the analyst to return to Step 1 to review and challenge 
the preliminary goals. In Step 7 when identifying and designing strategic alternatives, the 
analyst may need to revisit Step 2 and refine or add additional subsystems and actors. 
Again, when flagging uncertainties in Step 8, Step 2 may be revisited and revised. 
Finally, in Step 12 where the strategic alternatives are monitored for impacts and 
outcomes, it may be necessary to return to Steps 5, 7, 8, and/or 9. See Figure 20 for the 
iterative process. 
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1. Describe CLIOS System: 
Checklists & Preliminary 
Goal Identification 



2. Identify Subsystems in 
Physical Domain & Groups 
on Institutional Sphere 


3. Populate the Physical 
Domain & Institutional 
Sphere 


4A. Describe Components 


4B. Describe Links 


5. Transition from Descriptive to 
Prescriptive Treatment of System 


6. Refine CLIOS System 
Goals & Identify 
Performance Measures 


7. Identify & Design Strategic 
Alternatives for System 
Improvements 


8. Identify Important Areas of 
Uncertainty 


9. Evaluate Strategic 
Alternatives & Select 
"Bundles” 



Design and Implement Plan for: 

10. Physical Domain / 
Subsystems 

11. Institutional Sphere 


12. Evaluate, Monitors 
Adapt Strategic Alternatives 
for CLIOS System 



Figure 20. Iteration in the CLIOS Process (From: Sussman, 2007, p. 16) 
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IV. CLIOS STAGE 1 REPRESENTATION—MAJOR DEEENSE 

ACQUISTION SYSTEM 


A. INTRODUCTION 

The first stage of the CLIOS process—Representation—is the first order for 
understanding the CLIOS System, in this case the U.S. major defense acquisition system. 
The key functions of this stage are (a) to begin to understand the major defense 
acquisition system as a whole and (b) to establish preliminary goals for this system. The 
outputs for this stage are (1) system description, (2) issue identification, (3) goal 
identification, and (4) a structural representation. 

The CLIOS framework is an iterative process. As an analyst moves through the 
stages and obtains a greater understanding of the system, s/he will return to previous 
stages or steps to modify, refine, and improve. The complex nature of the CLIOS system 
requires this. 

This chapter presents a partial CLIOS application representing the physical 
domain of the U.S. major defense acquisition system nested within an institutional 
sphere. The first four steps of the CLIOS Process, which are found in Stage 1, are 
applied to the major defense acquisition system. 

B. STEP I—DESCRIBE THE SYSTEM: CHECKLISTS AND 

PRELIMINARY GOAL IDENTIFICATION 

Step 1 is the initial system description and preliminary goal identification. The 
CLIOS Process facilitates system description with a series of checklists for a high-level 
assessment. There are three checklists: Characteristics Checklist, Opportunities / Issues / 
Challenges Checklist, and Preliminary CLIOS System Goals Checklist. Refer back to 
Figure 17 on page 55 for more detail on checklists. These items may range from detailed 
descriptions to high level bullet points. The key idea in Step 1 is to initially frame the 
system. Although much of the literature review in Chapter II covers many of the items 
for the checklists in Step 1, below is a summation of the key points. 
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1 . 


Characteristics Checklist 


The Characteristics Checklist has a series of questions to facilitate the 
development of the checklists. 


a. What is the Temporal and Geographic Scale of the System? 


Characteristics Checklist - 
Temporal Scale: 


TIME SCALE 


> 


Military 

Real Time 

Tactical Planning 

Strategic Planning 

Industry 

Concept 

Detail Design 

Production 

Executive 
Office / 

Submit 

Budget 

Debate 

Approve / Reject 

Congress 

(for one aspect 
of the system) 



Eigure21. Temporal Scale 


Depending on whose perspective is used (see Eigure21), the time scale for 
the major defense acquisition system may be different. DoD spends three years of 
planning before submitting the request to be added to the President’s budget. Eor the 
defense industry, from concept to delivery, acquiring a major weapon system could take 
well over 10-15 years depending on the maturity level of technology. 
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Characteristics Checklist - 

Geographical Scale: GEOGRAPHIC 

SCALE 

- ^^^ ^ 

Local Regional National Global 


Eigure 22. Geographic Scale 


The weapon systems are designed and produced all over the U.S., 
including prime and subcontractors, vendors, program executive offices, stakeholders, 
etc. In addition, the actual weapons systems are utilized on a local, regional, national, 
and global basis (See Eigure 22 above). 

b. What are the Core Technologies and Systems? 


Characteristics Checklist continued - 

Some Core Technologies and Systems: 

• Software 

• Airborne Laser (ABE) 

• Laser 

• Global Hawk Unmanned Aircraft System 

• Radio 

• Kinetic Energy Interceptors 

• Physics 

• DDG 1000 Destroyer 

• Autonomous 

• E-2D Advanced Hawkeye 

• Physics 

• Terminal High Altitude Area Defense 
(THAAD) 

• And many more... 


Eigure 23. Core Technologies and Systems 


By 2007, DoD had 95 major defense acquisition programs in its portfolio 
(GAO, 2008). Major weapon systems include but are not limited to the list above. These 
systems require advanced technology. Years ago, DoD bought hardware, e.g., ships. 
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aircraft, etc., which were controlled by the warfighter. Today, the systems are advanced 
and complex with integrated software controlling most of the core subsystems within the 
platform. 


c. What are the Key Economic and Market Issues? 

Industrial base . Nearly 25 years ago, defense programs had over 20 prime 
contractors competing for multiple programs every year (DAPAP, 2005). Today there 
are four dominant prime contractors competing for fewer and fewer programs. 
Contractors have had to reduce plant capacity or merge with others. Although it is 
important for competition to play a role, fewer prime contractors and lower productions 
mean higher costs. Also, since one prime contractor cannot monopolize the market, 
contracts are usually shared because it is necessary to keep the workforce and industry 
knowledge current with more than one company. 

Workforce. Both the workforce producing the actual weapon systems and 
the workforce involved as military or civilians working the acquisition side are crucial to 
the system. Congressmen lobby for certain programs, which are produced in their states. 
The employment of their constituents is a key to economic development in their area. 

Inflation. Many of the GAO reports reviewed in Chapter II cited inflation 
as being a key concern for increase in cost estimates. 

d. Are There any Important Social or Political Issues or 
Controversies that Relate to this System? 

Pork barrel or Earmarks : As mentioned above under workforce, 
congressmen have an agenda that may be different from the need or the good of the 
whole. It is in their interest and that of their constituents to receive defense contracts for 
companies in their area or state for economic and/or sustainable development. Therefore, 
there is an incentive for them to lobby for programs that benefit them but may not benefit 
the nation as a whole. 

Unrequested weapons . Congress is known for adding items to the budget 
that were not necessarily requested by DoD. For example, this year (2009) Secretary of 
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Defense Robert Gates deleted the F-22 Raptor Air Dominance Fighters from the budget. 
However, Congress threatened to add them. 

Immature technology . Although there is a policy for not approving and 
funding projects until the technology has been demonstrated, programs continually 
receive funding with unproven and immature technology, requiring more funds later in 
the process and, of course, affecting schedule and performance. 

International security environment . The wars in Iraq and Afghanistan 
have demonstrated we are fighting a different kind of enemy that requires different 
strategies and weapons. 

e. What is the Historical Development and Context of the Major 
Defense Acquisition System? 

The U.S. acquisition system has been experiencing problems for more 
than a century and has been undergoing reform for more than half a century. The 
planning, programming, and budgeting system was instituted in 1962 when Secretary of 
Defense Robert McNamara decided to run the government like a business (Kausal, 2000). 
There has been a long history of inaccurate estimations, requirements creep, and poor 
program management. 

f What are the Policy Questions that Need to be Addressed? 

Funding . For stability, these major defense acquisition systems need long¬ 
term commitment, not year-to-year budget approval. Commitment to funding creates 
stability for the contractors, allowing them to focus on delivering capabilities rather than 
gaining incremental funding. This funding commitment to programs allows program 
managers to identify unknowns as high risks early without criticism or loss of funding. 

Technology. Although there are policies in place to avoid the risk of 
acquiring major defense systems without mature technologies, countless programs 
continue with unproven technology. 
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Oversight vs. Accountability . DoD has a complicated acquisition system 
emphasizing oversight of programs rather than emphasizing accountability by the 
contractors. 

Cost estimation . The process for estimating costs must be addressed. 
Whether the estimates are low to not cause “sticker shock” and gain initial approval or 
whether there is not enough knowledge of the program to understand the actual costs, the 
system must be improved. 

2. Opportunities, Issues, and Challenges Checklist 

This part of Step 1 captures aspects of the major defense acquisition system that 
later, in Stage 2, will be used to establish strategic alternatives. Information gathered in 
the Characteristics Checklist will be used as well. 

Cost, Performance, Schedule. Historically, cost and schedule have been an issue 
since with military acquisitions since the Civil War (DAPAP, 2005). 

Technology. The U.S. has been a leader in military technology. However, it 
costs money and time to develop these technologies. 

Cost. There is no standard, consistent and coherent cost tracking system that is 
actually used. 

Agendas . There are conflicting values among the acquisition community. As 
mentioned above, congressmen have an agenda that may be different from the need or the 
good of the whole. Industry would like to sell more programs and be paid to develop 
more advanced technology. The military branches have a desire for new, technologically 
advanced weapon systems. The U.S.’s fundamental strategy has been to deliver lethal 
violence beyond the enemies’ capability to withstand or bear. These advanced systems 
provide that capability. In addition, with the all-volunteer force, U.S. military branches 
are small. Therefore, the U.S. does not want to repeat the Korean or WW2 wars. The 
U.S. forces are currently stretched in Iraq. The advanced weapon systems enabled rapid 
defeat. However, holding on to “victory” nearly broke the U.S. The issues are not well 
resolved between political and military interests. 
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Threats . There are changes in the way war is handled-engaging different types of 
enemies requires different weapons and tactics. In the past, there were well-known, well- 
defined threats. Today, there is no single, well-defined threat. 

Funding / Stability . Although many of the programs require long-term 
commitment, the budgeting process is based on short-term decision making. Long-term 
cost increases are acceptable in order to achieve short-term budget savings. 

Economic. The major defense acquisition system is an economic engine 
employing hundreds of thousands across the nation, yet dependent on the economy. 

Value. In the past, the U.S. has focused on the continuous fielding of systems 
with the best possible performance, not necessarily focused on the best overall value for 
money. 


3. Preliminary Major Defense Acquisition System Goals Checklist 

. Provide the best, necessary capability to the warfighter to perform his/her 
job successfully for the best value. 

. Control cost, schedule, and performance. 

Step 1 begins to layout the full range of the system. The purpose of Step 1 is to 
highlight the main issues, identify important problems and possible causes, and provide a 
preliminary bounding of the system. This step helps determine what subsystems and 
components will be included, as well as important links. This is an iterative process. 
Later stages may require returning to stage one due to initially framing the system 
incorrectly. It is likely important aspects have been left out or only partially represented. 

C. STEP 2—IDENTIFY SUBSYSTEMS IN PHYSICAL DOMAIN & GROUPS 
ON INSTITUTIONAL SPHERE 

Step 2 of stage one requires the analyst to identify the major subsystems in the 
physical domain and the major actor groups on the institutional sphere. The major 
subsystems within the physical domain can be technical, natural, economic, social, and 
political. The major actor groups on the institutional sphere are the ones who are affected 
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by the subsystems within the physical domain, who attempt to influence those 
subsystems, and/or who “worry” about them (Sussman, 2007). 

For the physical domain, one logical approach to identifying subsystems is using 
the ones already defined by DoD: the Planning, Programming, Budgeting, and Execution 
(PPBE) system, the Joint Capabilities Integration and Development System (JCIDS), and 
the DoD 5000.02 Acquisition Process. 

Identification of the subsystems within physical domain: 

• Budget (PPBE) 

. JCIDS 

. Defense Acquisition System 

As stated in Chapter II , the PPBE is how the DoD allocates its resources. “It’s 
how DoD, and their contractors, manage to stay within their fiscal budget while they 
follow the Secretary of Defense’s policy, strategy, and goals” (DAU, 2009). The PPBE 
process overlaps with and supports the acquisition process by providing feedback and 
adjustments as needed during the execution phase. JCIDS key role is identifying required 
capabilities for the warfighters in support of national defense strategies. It is the 
mechanism for requirements generation and supports the Joint Requirements Oversight 
Council (JROC). The JCIDS process supports the acquisition and PPBE processes by 
identifying capability needs and associated performance criteria in order to develop and 
produce the necessary weapon systems. Einally, the defense acquisition system is the 
management process that guides all DoD acquisitions programs, including major defense 
programs. Eigure 24 demonstrates how these three entities interact for major weapon 
systems acquisition. Eor more detail, go the Defense Acquisition University’s website, 
https://acc.dau.mil/ifc/ . 
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Integrated Defense Acquisition, Technoiogy, and Logistics Life Cycle Management System 



Figure 24. Integrated Defense Acquisition (From: DAU website, https://acc.dau.mil/ifc/ ) 
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Although the physical domain has previously been identified and mapped by the 
Defense Acquisition University, these subsystems alone do not explain the complexity or 
describe the major defense acquisition system as a whole. There are major actor groups 
that have influence over or are influenced by the subsystems. 

Initial identification of major actor groups on the institutional sphere: 

. Congress 
. Industry 
. DoD 

. Military (Individual branches) 

. Executive Branch 
. U.S. public 
. Security Threats 

Due to the iterative nature of the framework, more major actor groups may be 
identified at other times during the CLIOS Process. 

D. STEP 3—POPULATE THE PHYSICAL DOMAIN & THE 

INSTITUTIONAL SPHERE 

Step 3 requires populating the physical domain and institutional sphere for the 
major defense acquisition system and identifying interactions between the two. See 
Figure 25. 

Each of the subsystems within the physical domain is layered and includes 
influences from the institutional sphere. There is overlap among the PPBE, JCIDS, and 
Acquisition subsystems. 
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Figure 25. Major Defense Acquisition System—Layering of Subsystems and 

Institutional Sphere 


Within the physical domain, there is overlap and links between the subsystems. 
Inside the institutional sphere, there is much interaction among the major actor groups. 
Furthermore, there are many links among the major actor groups in the institutional 
sphere and the major subsystems in the physical domain. See Figure 26 for a macro-level 
view of the links among the major actor groups within the institutional sphere and links 
among the major actor groups and the subsystems in the physical domain. 
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INSTITUTIONAL SPHERE 



Figure 26. Major Defense Acquisition System—Links Between Institutional Sphere and 

Physical Domain 
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E. STEP 4—DESCRIBE COMPONENTS AND LINKS 

In the final step of the representation stage, Step 4, the analyst describes the 
components in the physical domain, the actors in the institutional sphere, and the links 
among them. Within the physical domain, there are three subsystems: PPBE, JCIDS, 
and Defense Acquisition System. These three subsystems were designed to assist senior 
decision makers within the major defense acquisition system. They interact with each 
other, as well as operate separately. These three systems have been previously defined in 
both Chapter II and earlier in the fourth chapter. These subsystems have been identified 
by DoD as major decision-support systems. 

1. Physical Domain—Planning, Programming, Budgeting, and 

Execution (PPBE) 

The PPBE is the system used by the DoD to create their portion of the President’s 
budget. This is DoD’s strategic planning, program development, and resource allocation 
process. PPBE has links to both JCIDS and the acquisition system. It supports JCIDS 
with documents and training. It supports the acquisition system by understanding 
Security Threats and providing feedback during the execution phase. PPBE is a part of 
DoD that works with Military, Congress, and the President and employs U.S. civilians. 

2. Physical Domain—Joint Capabilities Integration and Development 
System (JCIDS) 

The Joint Chiefs of Staff, under DoD, created JCIDS to identify gaps in military 
needs and generate requirements. It interacts closely with the acquisition system to 
integrate capabilities in the acquisition system. The system is influenced by the Military 
and its needs, by Security Threats, and by Industry’s advancement of technology. The 
product influences Congress and the President’s decision concerning the need and 
funding for the major defense systems. 
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3. 


Physical Domain—Defense Acquisition System 


The Defense Acquisition System is the management process that guides all DoD 
major defense acquisition programs. It provides the policies that govern the defense 
acquisition programs. Major defense acquisition programs progress through a series of 
milestones associated with major phases of the program. It guides DoD personnel as well 
as Industry and Military. Congress is updated on the progress of these programs. 

The major actors within the institutional sphere of the major defense acquisition 
system are Congress, DoD, the Military, the Executive Branch, Industry, U.S. Public, and 
Security Threats. 

4. Institutional Sphere—Congress 

Congress has two major responsibilities when working within the major defense 
acquisition system: legislative process and the oversight function. Without Congress, 
there would be no major defense programs and no money to buy major weapon systems. 
Without major defense programs, there would be no national defense or jobs surrounding 
national defense, which includes military and civilians across the world. Both the Senate 
and the House have committees that influence DoD and the budgets. The Senate Armed 
Services Committee is responsible for a wide variety of policy and budgetary issues. The 
House Armed Services Committee has a wide range of responsibility including research 
and development in support of defense systems. This major actor group also functions as 
individuals. Each member of Congress is elected, and, usually, each has the desire to be 
re-elected—a prime motivator. Therefore, members may vote for or against items 
depending on the impact it has on his/her constituent’s opinion of the congressman—not 
on the impact of the nation as a whole. Alliances are formed among members of 
Congress to assist each others’ agendas. Congress is influenced by and influences 
Security Threats, the DoD, the President, Industry, the U.S. Public, and the Military. 


64 



5. 


Institutional Sphere—Department of Defense 


Although DoD is part of the Presidential Cabinet, it is considered a separate major 
actor group within the institutional sphere. The DoD basically has dual roles of 
responsibility: warfighting role and acquisition and logistics support role. It is DoD who 
procures and manages the major defense programs. Both the DoD and the military 
generate requirements for future military needs. All defense programs must be derived 
from documented and validated mission needs. For major defense programs, it is the 
Joint Chiefs of Staff who must approve the requirement for the program. However, 
Congress must give its approval for funding of the programs. Like Congress, DoD has 
much influence over acquisition policy. It is influenced by Congress, the President, the 
Military, Security Threats, and Industry. 

6. Institutional Sphere—The Military 

The military branches are not only the end users of the major defense systems, but 
they also play a key role in influencing the acquisition system. Each of the military 
departments is organized under the Secretary of Defense. They help with determining 
mission needs and generating requirements for major defense programs. The Military 
works closely with Industry, giving out funding for research of advanced technology and 
overseeing new or existing programs. The Military influences Industry, educating them 
on needs and requirements. On the other hand. Industry influences the Military 
suggesting new or different technology they believe would be better for the Military in an 
effort to shape the requirements. The Military anticipates and responds to Security 
Threats. 

7. Institutional Sphere—The Executive Branch 

The President is the Commander-in-Chief of the armed forces. However, it is 
Congress who has the power to declare war, determines the size of the military, creates 
policy that govern the military, and gives approval for funding major defense systems. 
The President submits a budget, which includes major defense programs, to Congress. 
After much discussion. Congress either approves with changes or disapproves. However, 
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the President has the final voice of vetoing or signing these bills. The Executive Branch 
works closely with DoD, since DoD is part of the Presidential Cabinet. The President 
appoints the head of the DoD, the Secretary of Defense. As witnessed earlier this year in 
2009, the Secretary of Defense has much influence over Congress, where Defense 
Secretary Gates warned Congress of spending money on unnecessary weapons. The 
President backed the Secretary of Defense saying he would veto any bill by Congress 
with funding for F-22s—an example of the interactions among these major actor groups 
within the institutional sphere. 

8. Institutional Sphere—Industry 

Industry plays a large role in the major defense acquisition system. It is industry 
that designs and produces the major defense weapons, employs hundreds of thousands of 
U.S. citizens to manufacture these weapons, and seeks to acquire more defense contracts 
for profit. Industry spends a portion of its own money in developing new technologies to 
shape the requirements of major defense systems. It lobbies to Congress in support of the 
systems they produce. Industry keeps abreast of Security Threats and designs solutions 
for DoD and the Military. 

9. Institutional Sphere—The U.S. Public 

The U.S. public plays a smaller role in the major defense acquisition system 
concerning day-to-day decisions on requirements, numbers, funding, etc. However, they 
still have influence on the system as a whole. One of the areas of influence is the many 
U.S. citizens who are employed as part of the defense-contracting arena or the civilian 
acquisition workforce. The employment not only helps them as individuals, but it also 
stimulates economic development in their region. Another area of influence is their right 
to vote. It is the U.S. public that chooses both the President and the congressmen. The 
U.S. public is influenced by Security Threats. Many citizens are actively involved in 
trying to influence their congressman and are concerned about either too much military 
presence in other nations, not enough military presence, anti-war, pro-war, etc. With the 
wars in Iraq and Afghanistan, many families have been affected by members serving or 
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dying in these wars and all of the other social implications surrounding it. The U.S. 
public has a voice, and therefore influence, on the use of their tax dollars. 

10. Institutional Sphere—Security Threats 

Security threats have always played a key role in the major defense acquisition 
system. The Cold War is a great example of a threat that influenced the acquisition and 
production of multiple major defense systems. It is the goal of the U.S. to protect its 
interests. Thus, when security is threatened, the military is called on and equipped to 
fight the threat. 

F. CLIOS ANALYSIS 

Basically, there are four goals among the major actor group for acquiring major 
weapon systems: (1) national security through acquiring systems with the required 
capability; (2) economic growth as seen with employment, generation of technology, and 
foreign exchange earning thru imports; (3) economic stability by acquiring capable 
systems while keeping public expenditures within limits; and (4) continue to build or 
sustain relations with other nations (Kausal, 1999). Therefore, the question remains as 
follows: Is it possible to (a) buy weapon systems with (b) enhanced capabilities at an (c) 
inexpensive price for financial stability and within a budget during a (d) reasonable 
amount of time while (e) generating jobs and useful technology in both the defense and 
civil sectors, which still (f) sustains or strengthens relations with other nations? 

Past answers to this question, and solutions to this complex problem, have 
involved compromise. Future answers most likely will continue to include compromise 
on one or more of the elements involved. However, with the insight gained through the 
CLIOS Process, a more holistic approach can be taken to identifying the real problems 
with the system and implementing solutions. Although only the Representation Stage has 
been applied to the major defense acquisition system, preliminary analysis can begin 
using the knowledge obtained of the subsystems in the physical domain and the major 
actor groups within the institutional sphere. Obviously, it is necessary to proceed through 
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the iterative CLIOS process to understand, validate, refine, or modify. Nevertheless, with 
this current knowledge, two observable problems with corresponding solutions are 
presented. 


1. Observation I: Long-term Decisions—Short-term Information 

From the CLIOS analysis, one of the problems appears to be long-term decisions 
being made on short-term information. The problem illustrated in Figure 27 shows long¬ 
term system acquisition decisions made in an environment utilizing short-term 
information. The result is a tendency for the institutions below the dashed line to 
underestimate future costs. 

The literature review conducted in Chapter II repeatedly revealed funding and 
production decisions being made on programs where realistic technology performance 
was unknown. Garvin and Roberto (2001) said, "facts come in two varieties: those that 
have been carefully tested and those that have been merely asserted or assumed” (p. 32). 
Due to the time period imposed, there is not enough time to carefully test. 

In their research on why people or groups make decisions without the necessary 
knowledge, Cho, Jerrell, and Landay (2000) identified seven barriers to knowledge-based 
decisionmaking. The barriers identified apply to the weapons defense acquisition system: 
(1) Limited understanding of the total customer need; (2) Political/financial instability; 
(3) No time to share insights or solutions; (4) Culture Clash, (5) Process/cultural 
complexity; (6) Ineffective or nonexistent communication; and (7) Geographic dispersion 
of workers (p. 2-7). In order to have knowledge of a particular subject, a person 
undergoes a sequence of steps. In the beginning, data is gathered from dispersed 
elements. Once a pattern is seen in the data, then information is obtained. However, a 
person or group does not have knowledge until both the information plus experience 
come together. The current cycle for making decisions, does not allow enough time to 
make these important decisions based on real knowledge. 
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Figure 27. CLIOS Representation: Long-term Decisions - Short-term Information 

2. Solution I: Integrated Process Team 

A possible strategic alternative to the current operation is an Integrated Process 
Team. Congress, JCIDS and the Defense Acquisition System would form an integrated 
process team jointly considering both near and far-term economic conditions and security 
threats before new long-term defense programs were approved and funded. See Figure 28 
for an illustrated depiction. 
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Figure 28. Integrated Process Team 

Of course, there are barriers and challenges to an idea such as this, one being the 
challenge of implementing such a team. It is important to bring together the right people 
to work together as a team on this problem. The idea of an integrated team is not new. 
France has Integrated Program Teams that includes the individual services, DGA, which 
is the DoD equivalent, and industry. Germany has Integrated Project Teams, which bring 
together all stakeholders including industry except during competition. The UK has 
Integrated Project Teams where responsibility and accountability are combined to focus 
on the customer and achieve seamless flow of responsibility from concept to disposal. 
Like Germany, the UK involves all stakeholders including industry except during 
competition. The U.S. uses many Integrated Product Teams, each focusing on a different 
product. 
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3. 


Solution II; Mandated Systems Engineering Organization 


Another possible strategic alternative is to have a new federally mandated systems 
engineering organization. This systems engineering organization would consider 
economic and threat conditions, analyze technological maturity, and make independent 
cost estimates as inputs to new defense system acquisition decisions. It would be equally 
accountable to Congress and DoD, not just a DoD office as the one stood up in early 
2009. If the technology is immature, the federal systems engineering organization would 
deter mi ne requirements for technology development programs. Industry would be 
awarded contracts, not on the basis of lowest cost, but rather on the ability to meet the 
federal systems engineering requirements. This alternative does not require formation of 
an integrated team. 



Figure 29. Mandated Federal Systems Engineering Organization 
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4. 


Observation II: Working in Silos—Little Accountability 


Another observation is there are times when major actor groups make decisions or 
work in silos. Each of the major actor groups is not “evil.” They work independently for 
causes, which, essentially, are not incongruous. However, when programs break down, 
fingers are pointed and blame is assigned, while it is hard to determine what went wrong 
and where. These independent actions facilitate inconsistency throughout the life cycle 
of the program. This results in little accountability for the overall problems of the whole 
system. See Figure 30 for a depiction of this. 
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Further along it is easy to blame 
industry, but problems may have 
been instigated from decisions made 
by other actor groups 


Figure 30. CLIOS Representation: Working in Silos - Little Accountability 


5. Solution III: Defense Acquisition Accountability 


This final possible strategic alternative is to have a hybrid of the Integrated 
Process Team and the Federal Systems Engineering organization. This hybrid would be a 
program-based congressional, DoD and industry coalition resulting in each partner 
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economically penalized for large overruns, schedule slips, or other issues within a 
program. All will be held accountable by the independent. Federal Systems Engineering 
organization. See Figure 31. 


Accountability would have to be instilled at every level in order for consistency 
and stability to thrive within the program. Changes will include concrete requirements, 
full commitment to funding, and minimized time from order to delivery. 



Figure 31. Defense Acquisition Accountability 
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G. CONCLUSION 

The first stage of the CLIOS Process, the Representation Stage, was applied to the 
major defense acquisition system. Essentially, there are four steps within this stage. 
Each step was discussed concerning the major defense acquisition system. The outputs 
of this stage rendered a better understanding of the subsystems within the whole system 
as well as the major actor groups influencing the system. As this system continues to 
progress through the other stages of the CLIOS Process, the iterative nature of the 
process will require returning to the Representation Stage and modifying or refining as 
necessary. 

Despite continuing with the CLIOS Process, initial analysis was conducted based 
on the observations of Stage 1. This resulted in two observations: (1) Long-term 
decisions made on short-term information and (2) Independent working leads to lesser 
accountability. Three strategic alternatives were identified: (1) form an Integrated 
Process Team with Congress, JCIDS and the Defense Acquisition System to jointly 
consider both near and far-term economic conditions and security threats before new 
long-term defense programs were approved and funded; (2) form a mandated Lederal 
Systems Engineering organization equally accountable to both Congress and DoD to 
consider economic and threat conditions, analyze technological maturity, and make 
independent cost estimates as inputs to new defense system acquisition decisions; and (3) 
create a hybrid of the two previous solutions to implement accountability at all levels of 
the acquisition to all major actor groups. Of course, challenges to implementation will 
exist. However, as stated above, continuing with the CLIOS Process will facilitate in 
refining the strategic alternatives, as well as implementing them. 
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V. CONCLUSION 


A. CONCLUSION 

Analyzing the major defense acquisition system is a challenging, complex 
process. The system has been plagued with problems since the beginning. Hundreds of 
reports were written on these issues and recommendations for improvement have been 
plentiful. Although social science research such as this usually concludes with direct 
recommendations for changing public policy, this research is limited to providing a better 
foundation for understanding the system. The goal of this research was to improve 
understanding of the intricacies of the U.S. major defense acquisition system for framing 
future solutions. The CLIOS Process provides such a framework for understanding a 
complex, large-scale, interconnected, open, sociotechnical system. 

B. SUMMARY OF KEY POINTS 

The system surrounding major weapons acquisitions continues to produce 
unwanted results: enormous cost overruns, unsatisfactory schedule delays, and decreased 
performance. The U.S. major defense acquisition system is very complex with many 
stakeholders, decision makers, and outside influences. The major defense acquisition 
system is not influenced by DoD and Congress only. It is subject to the President’s 
approval, the requirements of the military, the capability of industry, the security threats 
from other groups or states, and the opinions of the American public. 

Although policies are in place to thwart the progression of a program with 
unproven technology. Congress continues to approve budget for these programs with 
undemonstrated technology resulting in increased costs, schedule delays, and lesser 
performance than anticipated. Congress adds to the issues by not committing long-term 
to programs that do have proven technology. The funding is released incrementally 
without assurance for the future. Congress continues to add items to the DoD budget, 
which were never requested by DoD. Yet, it slashes other programs, which were 
requested based on identified mission needs and requirements. On the other hand, DoD 
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continues to ask for weapons systems for wars that are no longer being fought, not 
recognizing the U.S. has a different enemy to fight with different capabilities needed to 
be successful. 

Industry, along with other associated acquisition personnel, is continually blamed 
for insufficient management or mismanagement in general. There are very few prime 
contractors competing for very few major defense weapons acquisition programs. In 
order to keep contractors “alive” and to keep a sizeable and knowledgeable defense 
workforce, many contracts are split and shared between two or three. 

C. LESSONS LEARNED / HEURISTICS 

There are hundreds of quality, valuable reports available to better understand the 
system and influences. GAO continues to report on major defense acquisitions and 
continues to make recommendations. However, it appears that either no one takes heed, 
or policies are created but not implemented. 

What is best for the nation is not always the agenda of the stakeholders. 
Congressmen continue to lobby for programs that are best for their region and 
constituents but not necessarily best for the current U.S. military needs. The military 
continue to ask for more than originally required or funded causing requirements creep. 
Industry shapes the military requirements based on their products. 

The best technology has become the enemy of good enough technology. 
Advanced, unproven technology has been an issue with most major defense weapon 
system acquisitions. The U.S. is the dominant military in the world with the most 
advanced technology. The U.S. spends more than any other country. Some of the 
advanced technology being sought is not necessary to fight and win against the threats 
faced by the U.S. today. 

Accountability must be injected in the acquisition culture. There are “too many 
hands in the pot” to know who is accountable for what. In terms of the CLIOS Process, 
there are too many major actor groups with influences on the system. Everyone involved 
in the acquisition process—Congress, DoD, the Military, Industry—must assume a sense 
of accountability to the U.S. 
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The discipline of systems engineering has already defined guidelines for program 
success: Requirements management and definition; Systems architecture development; 
System, subsystem design; Systems integration and design; Validation and verification; 
System development and post deployment; etc. These concepts should be practiced by 
all stakeholders, and major actor groups, within the major defense acquisition system. 
Although DoDD 5000.02 mandates systems engineering on programs, it seems to apply 
to the industry contractors. Systems’ thinking is not prominent in all aspects of the 
system. Everyone involved in decision making concerning major defense acquisition 
must understand systems engineering and must employ a systems thinking approach. 

The CLIOS Process unveils the complexity of the system. Utilizing the CLIOS 
framework emphasizes the complexity of the U.S. major defense acquisition system even 
more. The Representation Stage is just the beginning of wading through the subsystems 
and actor groups. Although applying the entire CLIOS Process to the U.S. major defense 
acquisition system will take time, it will uncover more points of leverage for 
improvement and allow for real changes to be implemented. 

D. RECOMMENDATIONS AND FURTHER RESEARCH 

Further research should be conducted on the U.S. major defense acquisition 
system using the CLIOS Process. The remaining stages of the CLIOS Process, especially 
the iterative aspects of the process, should be applied to the U.S. major defense 
acquisition system. Much more will be discovered about the system, its behavior, and the 
leverage points for strategic alternatives. As the application of the CLIOS Process 
continues, deeper understanding of the system will be obtained. Stage 1 allowed for the 
initial understanding of the system, as well as provided the initial mental mapping of 
links within the physical and institutional systems. This stage produced general insights 
into the U.S. defense acquisition system. However moving into Stage 2, the analyst will 
gain a detailed and quantitative understanding of the system. Stage 2 will provide greater 
understanding and appreciation for system possibilities, limits, uncertainties, and 
sensitivities. Because of the newer comprehension during Stage 2, the analyst will go 
back to Stage 1 and modify and refine the initial structure—an iterative process. Finally, 
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implementing strategic alternatives in Stage 3 will cause changes within the physical 
domain and the institutional sphere surrounding the major defense acquisition system. 
These changes must be evaluated, monitored, and adapted, as needed for improvement to 
the system as a whole. 
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